FLOSS Project Planets

Python Software Foundation: Au revoir PyCon Pune

Planet Python - Thu, 2017-09-14 16:42

By Anwesha Das

February 2017 marked the beginning of a new journey for a new regional Python Conference - PyCon Pune.

PyCon is the meeting place for community. It gives Pythonistas an opportunity to come out of the virtual world and meet the real people behind the nicknames and the handles. It gives them an opportunity to learn new things and share their knowledge with others.

Considering the vast geographical territory of India, a single PyCon event wasn’t sufficient. PyCon Pune offered the Python community another chance to interact.

It was a four-day event; the main conference on the first two days and development sprints the second two. It was a single-track event, so all 550 attendees could attend all the sessions.

The inside story:
Pune, also known as the Oxford of the East, is amongst the fastest growing cities in the Asia Pacific region. Pune witnessed this PyCon at a hotel called Amonora, the Fern. The venue was beautiful, and we were grateful to have it: just a few weeks before the conference began, we were afraid we would have no venue at all!

The event had been located at one of oldest engineering colleges in Pune, but the venue canceled two weeks prior to the conference. Not quite the kind of news any organizers would like to hear, last minute (I can vouch for that, I was one of them).

Python Software Foundation then extended its hand to help. Team PyCon Pune as well as the Python community in India cannot thank the PSF enough for this.

PyLadies in PyCon Pune:
The PSF has always been a huge supporter of PyLadies. This time, they offered to share their booth with us. PyLadies had a huge presence at the event. From volunteering, to management, PyLadies were there everywhere. Yes, “Python is for girls.” And if we PyLadies need support in the face of a crisis, the PSF is there to hold us.

United we stand: The first ever thing that we decided for the conference was the quote to be used on the conference t-shirt.

“Came for the language, stayed for the community”, by Brett Cannon
This set the tone of the conference. The conference is a completely volunteer-driven event: the website, finance, AV and the overall management were run by volunteers, celebrating the community in the truest sense. Help poured in from pythonistas worldwide. The logo was designed by Ryan Larch from Australia. Python communities from all over India along with Python Pune and PyLadies worked tirelessly together to make the event a success. These people keep the soul of Python (the community and language) alive.

Day 1 and Day 2 of the main conference:
With a welcome note by Kushal Das, the chair commenced the conference. The first keynote was by Honza Král. He talked about his journey in the open source world. It was interesting to know about his hurdles and how he overcame them. It is also always inspiring to see at masters were students once too. Next, Anand Chitipothu taught us to write beautiful code.The post-lunch session began with the keynote by the “official Perl guy” of the Python community, John “Warthog” Hawley, who described the path from software to hardware hacking.

The day ended with an enchanting experience - a keynote by Pravin Patil, a teacher who uses Python to teach Physics. Python plus Physics plus a Laser equaled magic in his presentation.

Katie Cunningham began Day 2, followed by the Economics Professor Stephen Turnbull. He has helped to develop Ghostscript, XEmacs, Python, and GNU Mailman. He offered a word of encouragement to us saying, “You can help develop Python - and you should!” After lunch, Nick Coghlan delivered his keynote. He discussed “opportunities and challenges in open collaboration.” The last keynote talk was by Terri Oda, about security in the open source world.

This led to the end of the main conference. The mentors for the dev sprints spoke on what they were going to work on. An open feedback session marked the close of the main conference.

Day 3 and Day 4 of the Developers Sprint
For sprints, the conference moved to the Red Hat office in Pune. A dozen projects added features and fixed bugs during the final two days of the conference. The sprints had proven to be the most popular portion of the conference: Tickets had sold out within a week.

The Red Hat office looked like a hackerspace over the weekend. People were coding, learning, having fun and celebrating Python. It was the first ever dev sprint experience for more than 95% of the attendees. It took most of the people to some time to understand what is going on and how they could participate.

Slowly folks started flocking around different mentors. There were good number of people gathering around Nick—many Pythonistas have a dream to become CPython core developers. More than 10 patches were submitted to the language. Web.py, ElasticSearch, Django, es-django-example, OpenCabs, Pagure and micropython held sprints as well. The actual number of patches submitted can be found here.

I took shelter in the micropython and hardware room, where we were worked on fun bunny boards with esp8266 devices. John was there guiding us, changing our lives (my life for certain) with blinking LEDs. The best surprise came at the end. He gave each of us a bunny board. What a lovely souvenir to take home!

The conference is intended to give people the feeling of community. The event is over but the spirit is hasn’t diminished. Please join us next year for PyCon Pune 2018, February 8 - 11.
Categories: FLOSS Project Planets

Plasma accessibility updates

Planet KDE - Thu, 2017-09-14 16:11

Marco Martin recently posted about some of the improvements in krunner, today I want to show some of the effort into navigating the Plasma panels.

This video shows a user navigating the plasma panel using voice and keyboard. A shortcut focusses the panel, and then one can use tab and cursor keys as normal. In future we will improve our key-focus visual indicators, and allow for richer interaction.

What makes Plasma different to existing apps

Plasma in general has been a sore point with regards to accessibility as it doesn’t follow some of the exact same concepts as a traditional toolkit. Some of these convention breakages are by design, in krunner you wouldn’t want to have to tab to a list of results in order to navigate results with the cursor keys. Unfortunately these changes, if done non-optimally, really conflict with core concepts of focus and screen reader parsing. On top of that, we have the issues of an emerging toolkit, which needs that extra push to get right.

Why is this work useful?

Blind people aside, the work here has multiple other advantages.

We need better keyboard navigation, regardless. You often hear people say they prefer console apps; it’s not because they’re inherently better, but because being “handicapped” forces them to have good keyboard handling. We should be matching or beating that.

Also, approaching a large code base from a completely different angle has helped to tidy up some complex code that has built up over the years. Kickoff key handling is now not only better but the code is 2/3 the size.

Categories: FLOSS Project Planets

FSF Blogs: Friday Free Software Directory IRC meetup: September 15th starting at 12:00 p.m. EDT/16:00 UTC

GNU Planet! - Thu, 2017-09-14 15:12

Participate in supporting the Directory by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on irc.freenode.org.

Tens of thousands of people visit directory.fsf.org each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the Directory has been and continues to be a great resource to the world for over a decade now, it has the potential to be a resource of even greater value. But it needs your help!

This week we're back to adding new packages to the Directory. We'll also be checking in on the import project, which will allow for the automated creation of entries from repositories based on their previously vetted licenses, and which could grow the Directory by a massive amount. The more the merrier! That goes for software packages, but also for friends joining us for the weekly meeting. Hope to see you all there.

If you are eager to help, and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly Directory Meeting pages that everyone is welcome to contribute to before, during, and after each meeting.

Categories: FLOSS Project Planets

Friday Free Software Directory IRC meetup: September 15th starting at 12:00 p.m. EDT/16:00 UTC

FSF Blogs - Thu, 2017-09-14 15:12

Participate in supporting the Directory by adding new entries and updating existing ones. We will be on IRC in the #fsf channel on irc.freenode.org.

Tens of thousands of people visit directory.fsf.org each month to discover free software. Each entry in the Directory contains a wealth of useful information, from basic category and descriptions, to providing detailed info about version control, IRC channels, documentation, and licensing info that has been carefully checked by FSF staff and trained volunteers.

While the Directory has been and continues to be a great resource to the world for over a decade now, it has the potential to be a resource of even greater value. But it needs your help!

This week we're back to adding new packages to the Directory. We'll also be checking in on the import project, which will allow for the automated creation of entries from repositories based on their previously vetted licenses, and which could grow the Directory by a massive amount. The more the merrier! That goes for software packages, but also for friends joining us for the weekly meeting. Hope to see you all there.

If you are eager to help, and you can't wait or are simply unable to make it onto IRC on Friday, our participation guide will provide you with all the information you need to get started on helping the Directory today! There are also weekly Directory Meeting pages that everyone is welcome to contribute to before, during, and after each meeting.

Categories: FLOSS Project Planets

Annertech: How to allow an Editor Choose the View Mode of an Entity Reference Field in Drupal 8

Planet Drupal - Thu, 2017-09-14 14:20
How to allow an Editor Choose the View Mode of an Entity Reference Field in Drupal 8

Say you are building a website which has a 'Related Content' feature. Then say your client says something like "This is great, but all the related content looks like the teaser on the listing page. Can't we choose ourselves how we want it to look?" What's your response? You say yes, and you go install Display Suite or Panels or some other heavy duty module? Or, say yes and follow these neat little instructions. No one says no to clients, do they?

Here's what you need to do:

Categories: FLOSS Project Planets

Enthought: Webinar: Python for MATLAB Users: What You Need To Know

Planet Python - Thu, 2017-09-14 14:00

What:  A guided walkthrough and Q&A about how to migrate from MATLAB® to Python with Enthought Lead Instructor, Dr. Alexandre Chabot-Leclerc.

Who Should Watch: MATLAB® users who are considering migrating to Python, either partially or completely.


Python has a lot of momentum. Many high profile projects use it and more are migrating to it all the time. Why? One reason is that Python is free, but more importantly, it is because Python has a thriving ecosystem of packages that allow developers to work faster and more efficiently. They can go from prototyping to production to scale on hardware ranging from a Raspberry Pi (or maybe micro controller) to a cluster, all using the same language. A large part of Python’s growth is driven by its excellent support for work in the fields of science, engineering, machine learning, and data science.

You and your organization might be thinking about migrating from MATLAB to Python to get access to the ecosystem and increase your productivity, but you might also have some outstanding questions and concerns, such as: How do I get started? Will any of my knowledge transfer? How different are Python and MATLAB? How long will it take me to become proficient? Is it too big a of a shift? Can I transition gradually or do I have to do it all at once? These are all excellent questions.

We know people put a lot of thought into the tools they select and that changing platforms is a big deal. We created this webinar to help you make the right choice.

In this webinar, we’ll give you the key information and insight you need to quickly evaluate whether Python is the right choice for you, your team, and your organization, including:

  • How to get started
  • What you need in order to replicate the MATLAB experience
  • Important conceptual differences between MATLAB and Python
  • Important similarities between MATLAB and Python: What MATLAB knowledge will transfer
  • Strategies for converting existing MATLAB code to Python
  • How to accelerate your transition


Presenter: Dr. Alexandre Chabot-Leclerc, Enthought Lead Instructor

Ph.D, Electrical Engineering, Technical University of Denmark


Python for Scientists & Engineers Training: The Quick Start Approach to Turbocharging Your Work

If you are tired of running repeatable processes manually and want to (semi-) automate them to increase your throughput and decrease pilot error, or you want to spend less time debugging code and more time writing clean code in the first place, or you are simply tired of using a multitude of tools and languages for different parts of a task and want to replace them with one comprehensive language, then Enthought’s Python for Scientists and Engineers is definitely for you!

This class has been particularly appealing to people who have been using other tools like MATLAB or even Excel for their computational work and want to start applying their skills using the Python toolset.  And it’s no wonder — Python has been identified as the most popular coding language for five years in a row for good reason.

One reason for its broad popularity is its efficiency and ease-of-use. Many people consider Python more fun to work in than other languages (and we agree!). Another reason for its popularity among scientists, engineers, and analysts in particular is Python’s support for rapid application development and extensive (and growing) open source library of powerful tools for preparing, visualizing, analyzing, and modeling data as well as simulation.

Python is also an extraordinarily comprehensive toolset – it supports everything from interactive analysis to automation to software engineering to web app development within a single language and plays very well with other languages like C/C++ or FORTRAN so you can continue leveraging your existing code libraries written in those other languages.

Many organizations are moving to Python so they can consolidate all of their technical work streams under a single comprehensive toolset. In the first part of this class we’ll give you the fundamentals you need to switch from another language to Python and then we cover the core tools that will enable you to do in Python what you were doing with other tools, only faster and better!

Additional Resources Upcoming Open Python for Scientists & Engineers Sessions:

Washington, DC, Sept 25-29
Los Alamos, NM, Oct 2-6, 2017
Cambridge, UK, Oct 16-20, 2017
San Diego, CA, Oct 30-Nov 3, 2017
Albuquerque, NM, Nov 13-17, 2017
Los Alamos, NM, Dec 4-8, 2017
Austin, TX, Dec 11-15, 2017

Have a group interested in training? We specialize in group and corporate training. Contact us or call 512.536.1057.

Learn More

Download Enthought’s MATLAB to Python White Paper

Additional Webinars in the Training Series:

Python for Scientists & Engineers: A Tour of Enthought’s Professional Technical Training Course

Python for Data Science: A Tour of Enthought’s Professional Technical Training Course

Python for Professionals: The Complete Guide to Enthought’s Technical Training Courses

An Exclusive Peek “Under the Hood” of Enthought Training and the Pandas Mastery Workshop

Download Enthought’s Machine Learning with Python’s Scikit-Learn Cheat Sheets

The post Webinar: Python for MATLAB Users: What You Need To Know appeared first on Enthought Blog.

Categories: FLOSS Project Planets

lightning @ Savannah: GNU lightning 2.1.1 released!

GNU Planet! - Thu, 2017-09-14 13:58

GNU lightning is a library to aid in making portable programs
that compile assembly code at run time.


Download release:

2.1.1 adds support for jit_allocar for runtime alloca, contrary to
jit_allocai that provides a static offset allocation.
2.1.1 also adds support for jit_va_start, jit_va_arg, jit_va_arg_d
and jit_va_end, that implements C compatible varargs functions.
Note: alpha, hppa, mips (n32 abi), ppc (Darwin abi) and sparc ports
are no longer supported due to lack of a test environment.


o Implement cache flush.
o Do not use a dangling pointer for double to integer copy.

o Correct issues in load and store in the x32 abi.

o Correct Fibonacci number generation examples.
o Correct wrong jit_movr simplification.
o Correct wrong live register information for some complex patterns.
o Adapt for binutils 2.29.

Categories: FLOSS Project Planets

PyCharm: PyCharm 2017.3 EAP Starts now: scientific mode, and more

Planet Python - Thu, 2017-09-14 13:30

At JetBrains, we believe it’s important to involve our customers at an early stage. Before releasing a new version, we test our software to make sure everything works well. However, in software development there are as many unique configurations as there are developers. Therefore we provide Early Access Preview (EAP) versions free of charge, so you can make sure all of our improvements work in your environment prior to release.

We’d like to invite you to download the first EAP version of PyCharm 2017.3 now, it’s available from our website.

Please let us know how you like it! You can submit feedback from within PyCharm, by leaving a comment on the blog, or by reaching out to us on Twitter. If you run into any issues in the EAP, please create a ticket on YouTrack.

New in this version: Scientific Mode

More and more Python developers are doing scientific development, and we’ve created a new feature especially for them. When we detect that you use a scientific package in your project (like numpy or pandas), we will suggest to enable scientific mode:

If you chose not to use scientific mode when we asked, you can always enable it later in View | Scientific Mode.

In this mode, we open several tool windows by default: the Python console (bottom left), a variable viewer (bottom right), and a documentation viewer (top right).

The new scientific mode makes exploring data a lot easier: you always have an overview of the variables in your current session, and we look up the documentation of the method you’re using as you’re typing. No longer will you need to have your browser open with 20 tabs just to find the details about Pandas methods. In other words: you can focus on your data, rather than the details of the library you’re using.

New Features
  • You can now configure exception breakpoints to be conditional, or to evaluate a statement and log

  • For those who want to install PyPI packages into Conda envs, we’ve added PyPI and Conda Modes for the PyCharm Package Manager. Simply click the Conda button on the right to switch between Conda and PyPI mode:

Further Fixes and Improvements
  • We now correctly identify test classes, even if they have methods that don’t start with test_
  • Many inspections and autocomplete fixes, for example we’ve improved a couple of cases where we misinterpreted __all__, and we resolved some NamedTuple issues
  • Bugs involving Twisted Trial
  • Some improvements to editing BDD feature and step files
  • Various performance improvements

To see the full list of improvements, see the release notes of PyCharm 2017.3 EAP 1

Excited to try these new features yourself? Download PyCharm 2017.3 EAP 1 now

As always, EAP versions of PyCharm Professional Edition are free of charge, and will work for 30 days from the day of the release. If you upgrade regularly during the EAP program, you will be able to use PyCharm Professional Edition for free for the entire duration.

-PyCharm Team
The Drive to Develop

Categories: FLOSS Project Planets

Randa Report: The Fall of KDateTime

Planet KDE - Thu, 2017-09-14 11:29

The main goal for me and Volker for this year Randa Meeting was to port KCalCore, our calendaring library, away from KDateTime. KDateTime/KTimeZone are classes for handling date, time and time zones which were used back in the KDE4 (and earlier) days when Qt’s QDateTime wasn’t good enough for us, mainly due to missing time zone support.

In Qt5 QDateTime finally gained all the necessary features we need which made it possible for us to leave KDateTime behind. But we couldn’t just go and replace “K” with “Q” everywhere – there are many subtle (and not-so-subtle) differences in API, behavior and some missing features of QDateTime that required us to carefully consider each step. The fact that we started working on this over 2 years ago shows how challenging task this was.

We did not expect to finish the port here, but once we dived into it, things went fairly well and I’m glad to say that after four days of intensive hacking, surrounded by Swiss Alps and mountains of chocolate, we succeeded. KCalCore is now free of KDateTime and KTimeZone and that in turn made (after some minor adjustments) the rest of KDE PIM free of KDELibs4Support. That’s a huge milestone for us.

Many thanks to John Layt for laying the initial ground for the porting and to Mario and Christian for the steady stream of chocolate and sweets to soothe our nerves :-)

If you want to help us to continue improving Kontact and other parts of KDE, please donate to the Randa fundraiser so that we can continue organizing similar productive sprints in the future. Thank you!

Categories: FLOSS Project Planets

KMarkdownWebView 0.1.0

Planet KDE - Thu, 2017-09-14 11:08

KMarkdownWebView 0.1.0 has been released.

The KMarkdownWebView software provides a KParts plugin for rendered display of Markdown files, using web technologies (webpage with JavaScript library which creates HTML from the plaintext handed in). This enables KParts-using applications (like the archiving tool Ark or the file manager Krusader) to show Markdown files in the target format. It is also prepared for the upcoming “Live Preview” plugin for KTextEditor-based applications like the editors/IDEs Kate & KDevelop (see introduction).

The separate library libKMarkdownWebView is done for sharing code with a future Markdown thumbnailer plugin, whose code is not yet committed to the repository.

KMarkdownWebView can be built both with QtWebEngine (preferred by buildsystem) and QtWebKit. Pass -DUSE_QTWEBKIT=TRUE to CMake to enforce the use of QtWebKit.

Download from: https://download.kde.org/stable/kmarkdownwebview/0.1.0/src/

sha256: 361fec6c4fa9396975e788cfe759b1c45a2ccfc2b83e64b5105869ef312a5410 kmarkdownwebview-0.1.0.tar.xz


The technology behind KMarkdownWebView could be considered rather a hack, to quickly solve the need for rendering Markdown files, with the following requirements in mind:

  • webpage-like layouting (single page adapting width to screen/window size)
  • chrome-less minimalistic UI
  • support of KParts stream API (as useful with “Live Preview” plugin for KTextEditor)

Ideally this would be implemented one day natively in Qt by e.g. using QTextDocument & friends. The Markdown Okular generator gets us already closer to this, but misses all those bullet-points for now.

Categories: FLOSS Project Planets

Mediacurrent: The Path(auto) Less Traveled

Planet Drupal - Thu, 2017-09-14 10:42
Building URL Aliases Based on Specific Conditions

Have you ever needed to generate URL aliases for an entity based on specific set of conditions? I was recently on a Drupal 8 project that needed the ability to generate custom URL aliases based on very specific criteria outlined by the client. Out of the box, Pathauto module was not flexible enough to handle the customization, but leveraging its API and providing my own hook implementation in a custom module allowed me to perform conditional checks on data and build the conditional URL structures.

Categories: FLOSS Project Planets

Help us create a privacy-focused Free software smartphone!

Planet KDE - Thu, 2017-09-14 10:02

The news is out! KDE and Purism are working together on a Free software smartphone featuring Plasma Mobile. Purism is running a crowdfunding campaign right now, and if that succeeds, with the help of KDE, the plan is to deliver a smartphone based on Plasma Mobile in January 2019.

Why do I care?

Data collection and evesdropping has become a very common problem. Not only governments (friendly and less-friendly) are spying on us, collecting information about our private lives, also companies are doing so. There is a lot of data about the average user stored in databases around the world that not only allows them to impersonate you, but also to steal from you, to kidnap your data, and to make your life a living hell. There is hardly any effective control how this data is secured, and the more data is out there, the more interesting a target it is to criminals. Do you trust random individuals with your most private information? You probably don’t, and this is why you should care.

Protect your data

Mockup of a Plasma Mobile based phoneMockup of a Plasma Mobile based phone[/caption]The only way to re-gain control before bad things happen is to make sure as little data as possible gets collected. Yet, most electronic products out there do the exact opposite. Worse, the market for smartphones is a duopoly of two companies, neither of which has as a goal the protection of its users. It’s just different flavors of bad.

There’s a hidden price to the cheap services of the Googles and Facebooks of this world, and that is collection of data, which is then sold to third parties. Hardly any user is aware of the problems surrounding that.

KDE has set out to provide users an alternative. Plasma Mobile was created to give users a choice to regain control. We’re building an operating system, transparently, based on the values of Free software and we build it for users to take back control.

Purism and KDE

In the past week, we’ve worked with Purism, a Social Purpose Corporation devoted to bringing security, privacy, software freedom, and digital independence to everyone’s personal computing experience, to create a mobile phone that allows users to regain control.
Purism has started a crowdfunding campaign to collect the funds to make the dream of a security and privacy focused phone.

Invest in your future

By supporting this campaign, you can invest not only into your own future, become an early adopter of the first wave of privacy-protecting personal communication devices, but also to proof that there is a market for products that act in the best interest of the users.

Support the crowdfunding campaign, and help us protect you.

Categories: FLOSS Project Planets

Colm O hEigeartaigh: Securing Apache Hive - part IV

Planet Apache - Thu, 2017-09-14 08:02
This is the fourth in a series of blog posts on securing Apache Hive. The first post looked at installing Apache Hive and doing some queries on data stored in HDFS. The second post looked at how to add authorization to the previous tutorial using Apache Ranger. The third post looked at how to use Apache Ranger to create policies to both mask and filter data returned in the Hive query.

In this post we will show how Apache Ranger can create "tag" based authorization policies for Apache Hive using Apache Atlas. In the second post, we showed how to create a "resource" based policy for "alice" in Ranger, by granting "alice" the "select" permission for the "words" table. Instead, we can grant a user "bob" the "select" permission for a given "tag", which is synced into Ranger from Apache Atlas. This means that we can avoid managing specific resources in Ranger itself.

1) Start Apache Atlas and create entities/tags for Hive

First let's look at setting up Apache Atlas. Download the latest released version (0.8.1) and extract it. Build the distribution that contains an embedded HBase and Solr instance via:
  • mvn clean package -Pdist,embedded-hbase-solr -DskipTests
The distribution will then be available in 'distro/target/apache-atlas-0.8.1-bin'. To launch Atlas, we need to set some variables to tell it to use the local HBase and Solr instances:
  • export MANAGE_LOCAL_HBASE=true
  • export MANAGE_LOCAL_SOLR=true
Now let's start Apache Atlas with 'bin/atlas_start.py'. Open a browser and go to 'http://localhost:21000/', logging on with credentials 'admin/admin'. Click on "TAGS" and create a new tag called "words_tag".  Unlike for HDFS or Kafka, Atlas doesn't provide an easy way to create a Hive Entity in the UI. Instead we can use the following json file to create a Hive Entity for the "words" table that we are using in our example, that is based off the example given here:
You can upload it to Atlas via:
  • curl -v -H 'Accept: application/json, text/plain, */*' -H 'Content-Type: application/json;  charset=UTF-8' -u admin:admin -d @hive-create.json http://localhost:21000/api/atlas/entities
Once the new entity has been uploaded, then you can search for it in the Atlas UI. Once it is found, then click on "+" beside "Tags" and associate the new entity with the "words_tag" tag.

2) Use the Apache Ranger TagSync service to import tags from Atlas into Ranger

To create tag based policies in Apache Ranger, we have to import the entity + tag we have created in Apache Atlas into Ranger via the Ranger TagSync service. After building Apache Ranger then extract the file called "target/ranger-<version>-tagsync.tar.gz". Edit 'install.properties' as follows:
  • Set TAG_SOURCE_ATLASREST_DOWNLOAD_INTERVAL_IN_MILLIS to "60000" (just for testing purposes)
Save 'install.properties' and install the tagsync service via "sudo ./setup.sh". Start the Apache Ranger admin service via "sudo ranger-admin start" and then the tagsync service via "sudo ranger-tagsync-services.sh start".

3) Create Tag-based authorization policies in Apache Ranger

Now let's create a tag-based authorization policy in the Apache Ranger admin UI (http://localhost:6080). Click on "Access Manager" and then "Tag based policies". Create a new Tag service called "HiveTagService". Create a new policy for this service called "WordsTagPolicy". In the "TAG" field enter a "w" and the "words_tag" tag should pop up, meaning that it was successfully synced in from Apache Atlas. Create an "Allow" condition for the user "bob" with the "select" permissions for "Hive":
We also need to go back to the Resource based policies and edit "cl1_hive" that we created in the second tutorial, and select the tag service we have created above. Once our new policy (including tags) has synced to '/etc/ranger/cl1_hive/policycache' we can test authorization in Hive. Previously, the user "bob" was denied access to the "words" table, as only "alice" was assigned a resource-based policy for the table. However, "bob" can now access the table via the tag-based authorization policy we have created:
  • bin/beeline -u jdbc:hive2://localhost:10000 -n bob
  • select * from words where word == 'Dare';
Categories: FLOSS Project Planets

Colm O hEigeartaigh: Securing Apache Hive - part II

Planet Apache - Thu, 2017-09-14 07:47
This is the second post in a series of articles on securing Apache Hive. The first post looked at installing Apache Hive and doing some queries on data stored in HDFS. In this post we will show how to add authorization to the previous example using Apache Ranger.

1) Install the Apache Ranger Hive plugin

If you have not done so already, please follow the first post to install and configure Apache Hadoop and Apache Hive. Next download Apache Ranger and verify that the signature is valid and that the message digests match. Due to some bugs that were fixed for the installation process, I am using version 1.0.0-SNAPSHOT in this post. Now extract and build the source, and copy the resulting plugin to a location where you will configure and install it:
  • mvn clean package assembly:assembly -DskipTests
  • tar zxvf target/ranger-1.0.0-SNAPSHOT-hive-plugin.tar.gz
  • mv ranger-1.0.0-SNAPSHOT-hive-plugin ${ranger.hive.home}
Now go to ${ranger.hive.home} and edit "install.properties". You need to specify the following properties:
  • POLICY_MGR_URL: Set this to "http://localhost:6080"
  • REPOSITORY_NAME: Set this to "cl1_hive".
  • COMPONENT_INSTALL_DIR_NAME: The location of your Apache Hive installation
Save "install.properties" and install the plugin as root via "sudo -E ./enable-hive-plugin.sh". The Apache Ranger Hive plugin should now be successfully installed. Make sure that the default policy cache for the Hive plugin '/etc/ranger/cl1_hive/policycache' is readable by the user who is running the Hive server. Then restart the Apache Hive server to enable the authorization plugin.

2) Create authorization policies in the Apache Ranger Admin console

Next we will use the Apache Ranger admin console to create authorization policies for Apache Hive. Follow the steps in this tutorial to install the Apache Ranger admin service. Start the Ranger admin service via 'sudo ranger-admin start' and open a browser at 'http://localhost:6080', logging on with the credentials 'admin/admin'. Click the "+" button next to the "HIVE" logo and enter the following properties:
  • Service Name: cl1_hive
  • Username/Password: admin
  • jdbc.url: jdbc:hive2://localhost:10000
Note that "Test Connection" won't work as the "admin" user will not have the necessary authorization to invoke on Hive at this point. Click "Add" to create the service. If you have not done so in a previous tutorial, click on "Settings" and then "Users/Groups" and add two new users called "alice" and "bob", who we will use to test authorization. Then go back to the newly created "cl1_hive" service, and click "Add new policy" with the following properties:
  • Policy Name: SelectWords
  • database: default
  • table: words
  • Hive column: *
Then under "Allow Conditions", give "alice" the "select" permission and click "Add".

3) Test authorization with Apache Hive

Once our new policy has synced to '/etc/ranger/cl1_hive/policycache' we can test authorization in Hive. The user 'alice' can query the table according to our policy:
  • bin/beeline -u jdbc:hive2://localhost:10000 -n alice
  • select * from words where word == 'Dare'; (works)
However, the user 'bob' is denied access:
  • bin/beeline -u jdbc:hive2://localhost:10000 -n bob
  • select * from words where word == 'Dare'; (fails)
Categories: FLOSS Project Planets

Lior Kaplan: Public money? Public Code!

Planet Debian - Thu, 2017-09-14 05:30

An open letter published today to the EU government says:

Why is software created using taxpayers’ money not released as Free Software?
We want legislation requiring that publicly financed software developed for the public sector be made publicly available under a Free and Open Source Software licence. If it is public money, it should be public code as well.

Code paid by the people should be available to the people!

See https://publiccode.eu/ for the campaign details.

This makes me think of starting an Israeli version…

Filed under: Uncategorized
Categories: FLOSS Project Planets

Deeson: Thoughts on certification from the UK’s 1st Drupal 8 Grand Master

Planet Drupal - Thu, 2017-09-14 04:16

As Drupal specialists, we’re proud to be the largest Acquia Certified team in Europe. And last month our development manager Mark Pavlitski became the first person in the UK to achieve Drupal 8 Grand Master status!

This special recognition is awarded to best of the best Drupal Developers, and requires the participant to pass three exams: Acquia Certified Developer, Back-end Specialist, and Front-end Specialist.

In this post, Mark shares his insights into the certification process.

The exams

I started with the Drupal 8 Developer test, which is more general than the subsequent two, and covers Drupal site building, theming, module development and fundamental web concepts. 

Then I sat the Drupal 8 Front-end Specialist exam which, as the name implies, is focussed on front-end development and Drupal theming concepts. I found this the most challenging of the three, having had more back-end experience. But most of the questions are written in a way that will be familiar to an experienced Drupal developer.

Finally, I sat the Drupal 8 Back end specialist exam. I found this one more straightforward, given my experience, though still challenging at times.

My tips for other developers

Acquia’s certification tests take place on Kryterion’s WebAssessor platform. Officially it supports Macs, but I found I had various issues with the software. Although their support was very helpful, I ended up switching to a Windows laptop to take the tests.

All of the questions are scenario based, describing a Drupal development problem with multiple choice answers. There were a couple of typos and one or two ambiguously worded questions, but overall the tests are presented in a way that will make sense to any seasoned Drupal developer.

The results

I was pleasantly surprised by how quickly the test results appear on the Acquia certification registry. The test portal says the results will take a couple of weeks to appear, but in my case it was as quick as a few hours.

Overall my experience with the Acquia Certification programme was great. The tests were well structured, and challenging but not confusing. I’d definitely recommend certification as a way for Drupal businesses and professionals to validate their skills and experience.

Want to be part of the largest Acquia Certified team in Europe and get paid time to support your open source projects? We’re hiring.

Categories: FLOSS Project Planets

Python Bytes: #43 Python string theory, v2

Planet Python - Thu, 2017-09-14 04:00
<h1>Python Bytes 43</h1> <p>This episode is brought to you by Rollbar: <a href="https://pythonbytes.fm/rollbar"><strong>pythonbytes.fm/rollbar</strong></a></p> <p><strong>Brian #1:</strong> <a href="https://github.com/asottile/future-fstrings"><strong>future-fstrings</strong></a></p> <ul> <li>A backport of fstrings to python &lt; 3.6</li> <li>Include an encoding string the top of your file (this replaces the utf-8 line if you already have it)</li> <li>And then write python3.6 fstring code as usual!</li> </ul> <pre><code> # -*- coding: future_fstrings -*- thing = 'world' print(f'hello {thing}') </code></pre> <ul> <li>In action:</li> </ul> <pre><code> $ python2.7 main.py hello world </code></pre> <ul> <li>I’m still undecided if I like this sort of monkeying with the language through the encoding mechanism back door. </li> </ul> <p><strong>Michael #2:</strong> <a href="https://www.youtube.com/watch?v=js_0wjzuMfc"><strong>The Fun of Reinvention</strong></a></p> <ul> <li>Keynote from PyCon Israel</li> <li>David Beazley rocks it again</li> <li>Let’s take Python 3.6 features and see how far we can push them</li> <li>Builds an aspect-oriented constraint system using just 3.6 features</li> </ul> <p><strong>Brian #3:</strong> <a href="https://medium.com/@almeidneto/sound-pattern-recognition-with-python-9aff69edce5d"><strong>Sound Pattern Recognition with Python</strong></a></p> <ul> <li>Using<code>scipy.io.wavfile.read</code> to read a .wav file.</li> <li>Looking for peaks (knocks).</li> <li>Using minimum values to classify peaks, and minimum distance between peaks.</li> <li>This is an interesting start into audio measurements using Python.</li> <li>Would be fun to extend to some basic scope measurements, like sampling with a resolution bandwidth, trigger thresholds, pre-trigger time guards, etc.</li> </ul> <p><strong>Michael #4:</strong> <a href="https://www.python.org/dev/peps/pep-0550/"><strong>PEP 550: Execution Context</strong></a></p> <ul> <li>From the guys at <a href="http://magic.io"><strong>magic.io</strong></a></li> <li>Adds a new generic mechanism of ensuring consistent access to non-local state in the context of out-of-order execution, such as in Python generators and coroutines.</li> <li>Thread-local storage, such as <code>threading.local()</code>, is inadequate for programs that execute concurrently in the same OS thread. This PEP proposes a solution to this problem.</li> <li>A few examples of where Thread-local storage (TLS) is commonly relied upon: <ul> <li>Context managers like decimal contexts,<code>numpy.errstate</code>, and <code>warnings.catch_warnings</code>.</li> <li>Request-related data, such as security tokens and request data in web applications, language context for<code>gettext</code> etc.</li> <li>Profiling, tracing, and logging in large code bases.</li> </ul></li> <li>The motivation from <a href="https://github.com/magicstack/uvloop"><strong>uvloop</strong></a> is obviously at work here.</li> </ul> <p><strong>Brian #5:</strong> <a href="https://medium.com/@bfortuner/python-multithreading-vs-multiprocessing-73072ce5600b"><strong>Intro to Threads and Processes in Python</strong></a></p> <ul> <li>Beginner’s guide to parallel programming</li> <li>Threads and processes are both useful for different kinds of problems.</li> <li>This is a good quick explanation of when and where to use either. With pictures!</li> <li>Threads <ul> <li>Like mini processes that live inside one process.</li> <li>Share mem space with other threads.</li> <li>Cannot run simultaneously in Python (there are some workarounds), due to GIL.</li> <li>Good for tasks waiting on IO.</li> </ul></li> <li>Processes <ul> <li>Controlled by OS</li> <li>Can run simultaneously</li> <li>Good for CPU intensive work because you can use multiple cores.</li> </ul></li> </ul> <p><strong>Michael #6:</strong> <a href="https://www.pyfilesystem.org/"><strong>Alternative filesystems for Python</strong></a></p> <ul> <li>PyFilesystem: Filesystem Abstraction for Python. </li> <li>Work with files and directories in archives, memory, the cloud etc. as easily as your local drive.</li> <li>Uses <ul> <li>Write code now, decide later where the data will be stored</li> <li>unit test without writing real files</li> <li>upload files to the cloud without learning a new API</li> <li>sandbox your file writing code</li> </ul></li> <li>File system backends <ul> <li><a href="https://www.pyfilesystem.org/page/appfs/">AppFS</a> Filesystems for application data.</li> <li><a href="https://www.pyfilesystem.org/page/s3fs/">S3FS</a> Amazon S3 Filesystem.</li> <li><a href="https://www.pyfilesystem.org/page/ftpfs/">FTPFS</a> File Transfer Protocol.</li> <li><a href="https://www.pyfilesystem.org/page/memoryfs/">MemoryFS</a> An in-memory filesystem.</li> <li><a href="https://www.pyfilesystem.org/page/mountfs/">MountFS</a> A virtual filesystem that can <em>mount</em> other filesystems.</li> <li><a href="https://www.pyfilesystem.org/page/multifs/">MultiFS</a> A virtual filesystem that combines other filesystems.</li> <li><a href="https://www.pyfilesystem.org/page/osfs/">OSFS</a> OS Filesystem (hard-drive).</li> <li><a href="https://www.pyfilesystem.org/page/tarfs/">TarFS</a> Read and write compressed Tar archives.</li> <li><a href="https://www.pyfilesystem.org/page/tempfs/">TempFS</a> Contains temporary data.</li> <li><a href="https://www.pyfilesystem.org/page/zipfs/">ZipFS</a> Read and write Zip files.</li> <li>and more</li> </ul></li> </ul> <h2>Our news</h2> <p>Michael: switch statement extension to Python: <a href="https://github.com/mikeckennedy/python-switch"><strong>github.com/mikeckennedy/python-switch</strong></a></p>
Categories: FLOSS Project Planets

Talk Python to Me: #129 Falcon: The bare-metal Python web framework

Planet Python - Thu, 2017-09-14 04:00
Full featured web frameworks such as Django are great. But sometimes, living closer to the network layer is just the thing you need.
Categories: FLOSS Project Planets

pgcli: Release v1.8.0

Planet Python - Thu, 2017-09-14 03:00

Pgcli is a command line interface for Postgres database that does auto-completion and syntax highlighting. You can install this version using:

$ pip install -U pgcli Features:
  • Add fish-style auto-suggestion from history. (Thanks: Amjith Ramanujam)
  • Improved formatting of arrays in output (Thanks: Joakim Koljonen)
  • Don't quote identifiers that are non-reserved keywords. (Thanks: Joakim Koljonen)
  • Remove the ... in the continuation prompt and use empty space instead. (Thanks: Amjith Ramanujam)
  • Add conninfo and handle more parameters with c (issue #716) (Thanks: François Pietka)
Internal changes:
  • Preliminary work for a future change in outputting results that uses less memory. (Thanks: Dick Marinus)
  • Remove import workaround for OrderedDict, required for python < 2.7. (Thanks: Andrew Speed)
  • Use less memory when formatting results for display (Thanks: Dick Marinus).
  • Port auto_vertical feature test from mycli to pgcli. (Thanks: Dick Marinus)
  • Drop wcwidth dependency (Thanks: Dick Marinus)
Bug Fixes:
  • Fix the way we get host when using DSN (issue #765) (Thanks: François Pietka)
  • Add missing keyword COLUMN after DROP (issue #769) (Thanks: François Pietka)
  • Don't include arguments in function suggestions for backslash commands (Thanks: Joakim Koljonen)
  • Optionally use POSTGRES_USER, POSTGRES_HOST POSTGRES_PASSWORD from environment (Thanks: Dick Marinus)
Categories: FLOSS Project Planets

James McCoy: devscripts needs YOU!

Planet Debian - Wed, 2017-09-13 23:18

Over the past 10 years, I've been a member of a dwindling team of people maintaining the devscripts package in Debian.

Nearly two years ago, I sent out a "Request For Help" since it was clear I didn't have adequate time to keep driving the maintenance.

In the mean time, Jonas split licensecheck out into its own project and took over development. Osamu has taken on much of the maintenance for uscan, uupdate, and mk-origtargz.

Although that has helped spread the maintenance costs, there's still a lot that I haven't had time to address.

Since Debian is still fairly early in the development cycle for Buster, I've decided this is as good a time as any for me to officially step down from active involvement in devscripts. I'm willing to keep moderating the mailing list and other related administrivia (which is fairly minimal given the repo is part of collab-maint), but I'll be unsubscribing from all other notifications.

I think devscripts serves as a good funnel for useful scripts to get in front of Debian (and its derivatives) developers, but Jonas may also be onto something by pulling scripts out to stand on their own. One of the troubles with "bucket" packages like devscripts is the lack of visibility into when to retire scripts. Breaking scripts out on their own, and possibly creating multiple binary packages, certainly helps with that. Maybe uscan and friends would be a good next candidate.

At the end of the day, I've certainly enjoyed being able to play my role in helping simplify the life of all the people contributing to Debian. I may come back to it some day, but for now it's time to let someone else pick up the reins.

If you're interested in helping out, you can join #devscripts on OFTC and/or send a mail to <devscripts-devel@lists.alioth.debian.org>.

Categories: FLOSS Project Planets
Syndicate content