Import Python: ImportPython Issue 82

Planet Python - Tue, 2016-10-25 21:39

Worthy Read
Python has come a long way. So has job hunting.
Try Hired and get in front of 4,000+ companies with one application. No more pushy recruiters, no more dead end applications and mismatched companies, Hired puts the power in your hands.Sponsor
Machine Learning over 1M hotel reviews finds interesting insightsOn this tutorial we learned how to scrape millions of reviews, analyze them with pre-trained classifiers within MonkeyLearn, indexed the results with Elasticsearch and visualize them using Kibana. Machine learning makes sense when you want to analyze big volumes of data in a cost effective way. The code repository is here - https://github.com/monkeylearn/hotel-review-analysis

Mike Driscoll: Python 201: An Intro to mockThe unittest module now includes a mock submodule as of Python 3.3. It will allow you to replace portions of the system that you are testing with mock objects as well as make assertions about how they were used. A mock object is used for simulating system resources that aren’t available in your test environment. In other words, you will find times when you want to test some part of your code in isolation from the rest of it or you will need to test some code in isolation from outside services.

Altair: Declarative statistical visualization library for Python, based on Vega-Litepep8Altair is a declarative statistical visualization library for Python.

7 Django Development Best Practices Each Web Developer Must KnowdjangoSet up Persistent Database Connections, Turn Cached Loading on, Store the Sessions in Cache, Keep the Application and Libraries Separate, Store All Templates in One Place, Install HTML5 Boilerplate, Monitor and Control Processes using Supervisor.

DSF Code of Conduct committee releases transparent documentationcommunityToday we're proud to open source the documentation that describes how the Django Code of Conduct committee enforces our Code of Conduct. This documentation covers the structure of Code of Conduct committee membership, the process of handling Code of Conduct violations, our decision making process, record keeping, and transparency.

Why are some functions in python spelled with underscore, while some are not: setdefault, makedirs, isinstance?discussionI always wondered that. Here is a reddit discussion on the same.

Teaching an AI to write Python code with Python codeAIThis post is about creating a machine that writes its own code. More or less. Introducing GlaDoS Skynet Spynet. More specifically, we are going to train a character level Long Short Term Memory neural network to write code itself by feeding it Python source code. The training will run on a GPU instance on EC2, using Theano and Lasagne. If some of the words here sound obscure to you, I will do my best to explain what is happening.

Writing an API with Flask-RESTfulRESTThis article will go over the details of how to create a RESTful API with Flask and Flask-RESTful. In Part 1 we will go over the API basics and how to implement a simple API. In Part 2 we will expand into advanced use cases powered by Flask-RESTful. All code that will be show is readily available on this repository.

SciPy 2016 videos are upvideoRunning Python Apps in the Browser by Almar Klein was a pretty interesting talk for me. See what interest you. Youtube channel.

How to Create a Custom Django MiddlewaredjangoIn a nutshell, a Middleware is a regular Python class that hooks into Django’s request/response life cycle. Those classes holds pieces of code that are processed upon every request/response your Django application handles.

Mike Driscoll: An Intro to coverage.pycoverageCoverage.py is a 3rd party tool for Python that is used for measuring your code coverage. It was originally created by Ned Batchelder. The term “coverage” in programming circles is typically used to describe the effectiveness of your tests and how much of your code is actually covered by tests. You can use coverage.py with Python 2.6 up to the current version of Python 3 as well as with PyPy.

Ajax Website Tutorial with DjangodjangoIn this tutorial we'll see a trivial example of how to do a ajax website with django. Good for students looking to learn the basics of Django/Ajax and see how it works.

Check out the Python & Django channels available on Gitter.communityGitter is like slack for developers. They have active Python, Django channels. Have a look.

Introduction to Zipline in PythonPython has emerged as one of the most popular language for programmers in financial trading, due to its ease of availability, user-friendliness and presence of sufficient scientific libraries like Pandas, NumPy, PyAlgoTrade, Pybacktest and more. Zipline is a Python library for trading applications that powers the Quantopian service mentioned above. It is an event-driven system that supports both backtesting and live-trading. In this article we will learn how to install Zipline and then how to implement Moving Average Crossover strategy and calculate P&L, Portfolio value etc.

Upcoming Conference / User Group Meet
PyCon Australia 2016
PyCon APAC 2016
EuroScipy 2016
PyCon MY 2016
Python Unconference 2016
Kiwi PyCon
PyCon ZA 2016

PokemonGo-DesktopMap - 204 Stars, 36 ForkElectron App around PokemonGo-Map
PokemonGo-Map - 128 Stars, 55 ForkLive visualization of all the pokemon in your area
asyncpg - 69 Stars, 2 ForkA fast PostgreSQL Database Client Library for Python/asyncio
choronzon - 46 Stars, 16 ForkAn evolutionary knowledge-based fuzzer
zhihu-terminal - 42 Stars, 2 Forkzhihu-terminal using python2.7.
awesome-wagtail - 14 Stars, 1 ForkA curated list of awesome packages, articles, and other cool resources from the Wagtail community.
reddit_get_top_images - 10 Stars, 1 ForkGet top images from any subreddit
aiosmtpd - 6 Stars, 1 ForkA reimplementation of the Python stdlib smtpd.py based on asyncio.
delft - 6 Stars, 1 ForkA Python tool that automatically optimizes deep learning pipelines using genetic programming.
Categories: FLOSS Project Planets

Import Python: ImportPython Issue 81

Planet Python - Tue, 2016-10-25 21:39

Worthy Read
Preview the Python Serverless Microframework for AWSawsServerless computing is one of the most talked-about subjects among AWS customers. The AWS serverless offerings, AWS Lambda and Amazon API Gateway, make it possible for developers to create and run API applications with built-in, virtually unlimited scalability without managing any servers. Today the AWS Developer Tools team is excited to announce the preview of the Python Serverless Microframework for AWS. You can read Martin Fowler talking about the benefits of Serverless architecture http://martinfowler.com/articles/serverless.html#benefits

Continuous Delivery Tool
Automate from SCM to Production with BuildMaster! 100+ Integrations; Free for 5 Users.Sponsor
Reverse debugging for PythonpypyThe PyPy team is pleased to give you "RevPDB", a reverse-debugger similar to rr but for Python.

Facebook chat bot aka joke bot with django tutorialchatbotsI have decided to try to develop a chat bot which does only one thing. Send a random joke like the below one without an image irrespective of what the user types

Book review: “Two Scoops of Django: Best Practices for Django 1.8”book reviewThe book can be used as a reference of best practices and a cover-to-cover guide to best practices. I’ve done both and found it to be enjoyable, accessible, and educational when read cover-to-cover and a valuable reference when setting up a new Django project or doing general Django development. It covers a huge range of material.

Using Spectrum - Standalone Logging Server with Python. ReviewSpectrum is a standalone logging server plus log viewer with filtering capabilities. It scales to multiple logging streams with endpoint being a file residing on filesystem, REST API endpoint, Syslog, UDPStream, WebSocketStream.

Create python app and deploy in cloud for free.
Have an app idea and want to see if it’ll fly ? Create a new Python app, free with Azure App Service. Build web, mobile, and API apps that use Redis Cache, Search, or Content Delivery Network and more. Check it out.Sponsor
Build Your First Python and Django ApplicationtutorialDecent tutorial to get people started with Python and Django.

Building an Async Python 3 Serverless Application with DockerserverlessI will show you a simple way to build a “serverless” application and test it via Docker. When I refer to “serverless” I’m referring to the idea that the application is a short lived app, does its job, stops – just like AWS Lambda. I will create two applications each in their own project folders: serverless-app and serverless-web The serverless-app piece is the actual “serverless” piece of this, the web app will run as long as we want. I just gave them similar names to make it easier to keep the projects named closely but different enough to know what does what.

djangobot: Bridge between Slack and Django, via ChannelsbotDjangobot is a bridge between Slack and a Channels-enabled Django app. Specifically, it is a protocol server that produces and consumes messages for channels-based apps. It is built atop autobahn and twisted.

Visualizing relationships between python packagespypiI extracted co-occurence of top 3500 python packages in github repos using the the github data on BigQuery. I implemented the visualization force layout in d3 via the velocity verlet integration. I also clustered the graph using algorithms from python-igraph and updated it to http://graphistry.com/.

Hiding a payload in PNG files with PythonsecurityHow could I store files online, in plain sight, for free. Because who doesn’t like a good ‘ol game of hide and seek. But with files. On the internet. Hide files in plain sight. Allow them to be distributed via free public channels. E.g Twitter, Reddit, imgur.


Senior Python Developer at EPAMMexicoEPAM Systems, Inc. (NYSE:EPAM), a leading global product development and platform engineering services company, is focused on delivering results through best in-class software engineering, combined with innovative strategy, consulting and design capabilities. Responsible for many phases of the software development lifecycle including technical research, and requirements analysis for Python projects.

Upcoming Conference / User Group Meet
PyGotham 2016
EuroPython 2016
Python Northwest
PyOhio 2016
PyCon Australia 2016
PyCon APAC 2016
EuroScipy 2016
PyCon MY 2016

aq - 31 Stars, 0 ForkQuery AWS resources with SQL. aq allows you to query your AWS resources (EC2 instances, S3 buckets, etc.) with plain SQL.
sokChoGo - 24 Stars, 5 ForkTravel the pokemon world in your comfy bed, with your keyboards, not on your legs.
macOSLAPS - 19 Stars, 2 ForkA python script that will change a local adminsitrator password to a random generated password. Similar behavior to LAPS for Windows
Pokemon_Go_API - 13 Stars, 0 ForkPokémon GO API in Python
stdgif - 10 Stars, 0 ForkStandard output for gifs. Dumps gifs to stdout or creates shell scripts that can be sourced from a .bashrc or other shell initialization file.
pstack - 9 Stars, 1 ForkTool to dump python thread and greenlet stacks. pstack is to python as jstack is to java! It's a debug tool to print python threads or greenlet stacks.
csv-sql - 8 Stars, 0 ForkQuery your CSV files with SQL
PostMail - 6 Stars, 1 ForkA simple mail server which can let you send a email only sending a POST request
Categories: FLOSS Project Planets

Import Python: Using Spectrum - Standalone Logging Server with Python. Review

Planet Python - Tue, 2016-10-25 21:39
Debugging, Logging and Software Development “ If debugging is the process of removing software bugs, then programming must be the process of putting them in. ” Edsger Dijkstra

During development of new features and integration there of a developer can be found staring at log data as it pours on his screen for errors, observing variable values and if they are in sync with expectation. Developers spends more time confronting, debugging and fixing software issues during development of software rather then after it has been deployed ( hopefully ).

Spectrum is to logging what sqlite3 is to databases. It’s a standalone logging server plus log viewer with filtering capabilities. It scales to multiple logging streams with endpoint being a file residing on filesystem, REST API endpoint, Syslog, UDPStream, WebSocketStream.

Let us give Spectrum a spin and see how it works.

Create Account

Visit https://www.devspectrum.com/register/ and register your account. This includes buying spectrum right away and sadly there is no trial period. Disclaimer we were provided access to the software by Spectrum for this product review.

Download Build and Registration Keys

Once your account is created the dashboard https://www.devspectrum.com/dashboard/ lists everything you need to get started

a) Copy Registration code somewhere secure,

b) Click “Download Latest Release” and select the operating system of your choice i.e. Linux, Windows, OSX.

c) Once the build has been downloaded extract the archive. (We are using the Linux build for this article). Execute Spectrum binary and you are greeted with a popup for registration code. Enter your Registration code from step 1 and we are in business.

Spectrum allows developer to organize logging into projects. Each project can have multiple streams. Each logging stream in turn can accommodates multiple levels of logging. Well that’s quite a mouthful but below examples will clear it.

File Based Stream

To fans of doing tail -f logfile.log on terminal this is for you. You wouldn’t have to worry about scrollback buffer. Simply create an endpoint with File as an option and select the log file.

Syslog and File based Stream import logging import logging.handlers logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) handler = logging.handlers.SysLogHandler(address = '/dev/log') formatter = logging.Formatter('%(module)s.%(funcName)s: %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) def dummylogs(): logger.debug('this is debug') logger.critical('this is critical') if __name__ == '__main__': dummylogs()

The entry level barrier to get spectrum up and running is very low, that’s why I called it sqlite3 of databases. Having the ability to search is very helpful too. e.g. “Traceback” shows the Python errors while the program executed.

REST based Stream

Spectrum can run as a standalone rest server binding to localhost and a port number of your choice. Python program can stream log messages to the server using spectrum-python library installable using pip.

pip install spectrum-python

Before we dive into code, it’s worth noting that a simple JSON HTTP POST to http://IP:PORT is all that’s needed to log messages to Spectrum in REST mode.

import requests url = "http://localhost:9000" data = {"id": "863ef2c3ea134841b87072", "level": "ERROR", "message": "The daemon process has generated an error code 555", "number": 253, "sublevel": "DAEMON", "timestamp": "2016-02-22 11:12:39.873108" } headers = {'Content-type': 'application/json', 'Accept': 'text/plain'} r = requests.post(url, data=json.dumps(data), headers=headers)

as long as your json has the keys id, level, sublevel, message, number, sublevel and timestamp you will be able to successfully log your message.

import logging from spectrum.handlers import Spectrum as SpectrumAPI logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) spectrum = SpectrumAPI('Main-Process') formatter = logging.Formatter('%(module)s.%(funcName)s: %(message)s') spectrum.setFormatter(formatter) logger.addHandler(spectrum) def dummylogs(): logger.debug('this is debug') logger.critical('this is critical') if __name__ == '__main__': dummylogs() Conclusion Pros
  • Minimal Viable logger with a decent UI
  • Easy to install, integrate and use
  • Works out of the box for Python
  • Ability to accommodate N no of projects with N no of streams and loggers.
  • Search bar could have access to filters to toggle and not overlap the first line,
  • File based logging method froze and crashed on us couple of times. Had to remove .spectrum directory in home folder and reinstall. This could be attributed to the fact that spectrum is a chrome app and not good at keeping up with filesystem IO. The software was writing an MB worth of data to the log file per second.
Categories: FLOSS Project Planets

Import Python: ImportPython Issue 79

Planet Python - Tue, 2016-10-25 21:39

Worthy Read
Have an idea for a Python App ? Get FREE $200 in Azure credit to get started.
Create a new Python app, free with Azure App Service. Free $200 in Azure credit. Build web, mobile, and API apps that use Redis Cache, Search, or Content Delivery Network and more.Sponsor
Top 10 Django Projects Started in 2016djangoOne of the cornerstones of Django is that there’s a solid core that others can extend and share their extensions. So it’s good to celebrate the best of those. Here’s my list derived from this query on Github for projects started this year. Not sure about top but useful projects that one can use in personal or work projects/products. Have a look.

Python programming humourhumourConversation with Tech Support.

Supercharging Django ProductivitydjangoThese design patterns also solidified a mentality within our team that there is an internal standard of handling models and views. We can extend our base classes to tackle any business requirement. Our codebase is constantly evolving, both in new features and how we model the world. Django has proven to be a valuable tool that lets us aim big and iterate often. Some might call this an anti-pattern nevertheless a novel approach.

Distribute Tasks with Python Celery and RabbitMQceleryIn this tutorial, we are going to have an introduction to basic concepts of Celery with RabbitMQ and then set up Celery for a small demo project. At the end of this tutorial, you will be able to setup a Celery web console monitoring your tasks.

Python Projects for Kids - Book Reviewbook reviewThat's it :) ... Go read the review.

Python 2.7.12 releasedreleaseThe Python 2.7.x series has a new bugfix release, Python 2.7.12, available for download.

Djangorecipe: easy test coverage reportsdjangoCode coverage reports help you see which parts of your code are still untested. Yes, it doesn’t say anything about the quality of your tests, but at the least it tells you which parts of your code have absolute the worst kind of tests: those that are absent :-)

Your Django Story: Meet Anna MakarudzeinterviewAnna Makarudze lives in Harare, Zimbabwe, and was born and raised in Masvingo. She is an an ICT consultant as well as a Python/Django developer.

anymail/django-anymail: Django email backends and webhooks for Mailgun, Postmark, SendGrid, SparkPost and moredjangoAnymail integrates several transactional email service providers (ESPs) into Django, with a consistent API that lets you use ESP-added features without locking your code to a particular ESP. It currently fully supports Mailgun, Postmark, SendGrid, and SparkPost, and has limited support for Mandrill.

Seven article series dedicated on mining twitter datatwitterNote these articles were published a year back, currently trending on social media. Curating it because it's pretty interesting and fun.

Exploring the NFL Draft with Pythonsport analysisAfter reading a couple of posts by Michael Lopez about the NFL draft, I decided to recreate some of his analysis using Python (instead of R). Post being - https://statsbylopez.com/2016/05/02/the-nfl-draft-where-we-stand-in-2016/

Text Analysis, Markov Chains and Bible Quotes Generator.machine learningText analysis and patterns are very interesting topics for me. I think every author has their own unique, distinctive pattern. The way they build sentences and phrases, not only in terms of tone, but the subconscious choices of word pairings and sequences. How their sentences begin and end, and what they put in the middle; they leave their cerebral fingerprints on paper, blog posts or even FaceBook status updates. I think it is a very achievable, having enough base data, to identify the author Note - It's a trivial example but a good starting point for someone looking to learn/explore Markov chains.

Upcoming Conference / User Group Meet
Fox Valley Python User Group
STL Python
Python Meeting Düsseldorf
Sydney Python User Group
PyGotham 2016
EuroPython 2016
PyOhio 2016

GraphDash - 95 Stars, 7 ForkA web-based dashboard built on graphs and their metadata.
Daily-Reddit-Wallpaper - 77 Stars, 15 ForkChange your wallpaper to most upvoted image of the day on /r/wallpapers or from any other subreddit (on startup).
pipup - 19 Stars, 1 ForkBetter handling of Python pip's requirements.txt files
kiwi - 8 Stars, 1 ForkA simple DynamoDB ORM for python, based on boto2
pyscheduler - 8 Stars, 0 ForkCreate scheduled tasks at runtime easily (Django, Flask, Bottle etc.)
git_log_viewer - 7 Stars, 0 ForkGit log viewer
markovy - 6 Stars, 1 ForkPython module for Markov Chain algorithm
Categories: FLOSS Project Planets

Laura Arjona Reina: Rankings, Condorcet and free software: Calculating the results for the Stretch Artwork Survey

Planet Debian - Tue, 2016-10-25 16:11
We had 12 candidates for the Debian Stretch Artwork and a survey was set up for allowing people to vote which one they prefer.

The survey was run in my LimeSurvey instance, surveys.larjona.net. LimeSurvey  its a nice free software with a lot of features. It provides a “Ranking” question type, and it was very easy for allowing people to “vote” in the Debian style (Debian uses the Condorcet method in its elections).

However, although LimeSurvey offers statistics and even graphics to show the results of many type of questions, its output for the Ranking type is not useful, so I had to export the data and use another tool to find the winner.

Export the data from LimeSurvey I’ve created a read-only user to visit the survey site. With this visitor you can explore the survey questionnaire, its results, and export the data. URL: https://surveys.larjona.net/admin Username: stretch Password: artwork First attempt, the quick and easy (and nonfree, I guess) There is an online tool to calculate the Condorcet winner, http://www.ericgorr.net/condorcet/  The steps I followed to feed the tool with the data from LimeSurvey were these: 1.- Went to admin interface of lime survey, selected the stretch artwork survey, responses and statistics, export results to application 2.- Selected “Completed responses only”, “Question codes”, “Answer codes”, and exported to CSV. (results_stretch1.csv) 3.- Opened the CSV with LibreOffice Calc, and removed these columns: id    submitdate    lastpage    startlanguage 4.- Remove the first row containing the headers and saved the result (results_stretch2.csv) 5.- In commandline: sort results_stretch2.csv | uniq -c > results_stretch3.csv 6.- Opened results_stretch3.csv with LibreOffice Calc and “merge delimitors” when importing. 7.- Removed the first column (blank) and added a column between the numbers and the first ranked option, and fulfilled that column with “:” value. Saved (results_stretch4.csv) 8.- Opened results_stretch4.csv with my preferred editor and search and replace “,:,” for “:” and after that, search and replace “,” for “>”. Save the result (results_stretch5.csv) 9.- Went to http://condorcet.ericgorr.net/, selected Condorcet basic, “tell me some things”, and pasted the contents of results_stretch5.csv there. The results are in results_stretch1.html But where is the source code of this Condorcet tool? I couldn’t find the source code (nor license) of the solver by Eric Gorr. The tool is mentioned in http://www.accuratedemocracy.com/z_tools.htm where other tools are listed and when the tool is libre software, is noted so. But not in this case. There, I found another tool, VoteEngine, which is open source, so I tried with that. Second attempt: VoteEngine, a Free Open Source Software tool made with Python I used a modification of voteengine-0.99 (the original zip is available in http://vote.sourceforge.net/ and a diff with the changes I made (basically, Numeric -> numpy and Int -> int, inorder that works in Debian stable), here. Steps 1 to 4 are the same as in the first attempt. 5.- Sorted alphabetically the different 12 options to vote, and assigned a letter to each one (saved the assignments in a file called  stretch_key.txt). 6.- Opened results_stretch2.csv with my favorite editor, and search and replace the name of the different options, for their corresponding letter in stretch_key.txt file. Searched and replaced “,” for ” ” (space). Then, saved the results into the file results_stretch3_voteengine.txt 7.- Copied the input.txt file from voteengine-0.99 into stretch.txt and edited the options to our needs. Pasted the contents of results_stretch3_voteengine.cvs at the end of stretch.txt 8.-In the commandline ./voteengine.py <stretch.txt  > winner.txt (winner.txt contains the results for the Condorcet method). 9.- I edited again stretch.txt to change the method to shulze and calculated the results, and again with the smith method. The winner in the 3 methods is the same. I pasted the summary of these 3 methods (shulze and smith provide a ranked list) in stretch_results.txt If it can be done, it can be done with R… I found the algstat R package: https://cran.r-project.org/web/packages/algstat/index.html which includes a “condorcet” function but I couldn’t make it work with the data. I’m not sure how the data needs to be shaped. I’m sure that this can be done in R and the problem is me, in this case. Comments are welcome, and I’ll try to ask to a friend whose R skills are better than mine!
And another SaaS I found https://www.condorcet.vote/ and its source code. It would be interesting to deploy a local instance to drive future surveys, but for this time I didn’t want to fight with PHP in order to use only the “solver” part, nor install another SaaS in my home server just to find that I need some other dependency or whatever. I’ll keep an eye on this, though, because it looks like a modern and active project. Finally, devotee Well and which software Debian uses for its elections?  There is a git repository with devotee, you can clone it: https://vote.debian.org/~secretary/devotee.git/ I found that although the tool is quite modular, it’s written specifically for the Debian case (votes received by mail, GPG signed, there is a quorum, and other particularities) and I was not sure if I could use it with my data. It is written in Perl and then I understood it worse than the Python from VoteEngine. Maybe I’ll return to it, though, when I have more time, to try to put our data in the shape of a typicall tally.txt file and then see if the module solving the condorcet winner can work for me. That’s all, folks! (for now…) Comments You can coment on this blog post in this pump.io thread
Filed under: Tools Tagged: data mining, Debian, English, SaaS, statistics
Categories: FLOSS Project Planets

Jose M. Calhariz: New packages for Amanda on the works

Planet Debian - Tue, 2016-10-25 14:41

Because of the upgrade of perl, amanda is currently broken on testing and unstable on Debian. The problem is known and I am working with my sponsor to create new packages to solve the problem. Please hang a little more.

Categories: FLOSS Project Planets

Pantheon Blog: Fixing the Composer Global Command

Planet Drupal - Tue, 2016-10-25 14:38
One of the most commonly documented ways for a PHP command line tool to be installed is via the composer global require command. This command is easy to document and easy to run, which explains its popularity. Unfortunately, this convenient function has a darker side that can cause some pretty big problems.
Categories: FLOSS Project Planets

Bits from Debian: "softWaves" will be the default theme for Debian 9

Planet Debian - Tue, 2016-10-25 13:50

The theme "softWaves" by Juliette Taka Belin has been selected as default theme for Debian 9 'stretch'.

After the Debian Desktop Team made the call for proposing themes, a total of twelve choices have been submitted, and any Debian contributor has received the opportunity to vote on them in a survey. We received 3,479 responses ranking the different choices, and softWaves has been the winner among them.

We'd like to thank all the designers that have participated providing nice wallpapers and artwork for Debian 9, and encourage everybody interested in this area of Debian, to join the Design Team. It is being considered to package all of them so they are easily available in Debian. If you want to help in this effort, or package any other artwork (for example, particularly designed to be accessibility-friendly), please contact the Debian Desktop Team, but hurry up, because the freeze for new packages in the next release of Debian starts on January 5th, 2017.

This is the second time that Debian ships a theme by Juliette Belin, who also created the theme "Lines" that enhances our actual stable release, Debian 8. Congratulations, Juliette, and thank you very much for your continued commitment to Debian!

Categories: FLOSS Project Planets

Tag1 Consulting: Tag1 Quo: Finding a Place in the (Version) Universe, Part 2

Planet Drupal - Tue, 2016-10-25 12:33
sam Tue, 10/25/2016 - 09:33

When we left off last time, we’d assembled a definition of what versions are. Now, we’re going to dive into how we use them: comparing them to one another!

The general goal is straightforward enough: we want to know if, say, 6.x-1.0 is less than 6.x-1.1. (Yup!) Or if 6.x-1.0-alpha1 is less than 6.x-1.0. (Also yup!) Let’s rewrite these two examples as tuple comparisons:

Categories: FLOSS Project Planets

Julian Andres Klode: Introducing DNS66, a host blocker for Android

Planet Debian - Tue, 2016-10-25 12:20

I’m proud (yes, really) to announce DNS66, my host/ad blocker for Android 5.0 and newer. It’s been around since last Thursday on F-Droid, but it never really got a formal announcement.

DNS66 creates a local VPN service on your Android device, and diverts all DNS traffic to it, possibly adding new DNS servers you can configure in its UI. It can use hosts files for blocking whole sets of hosts or you can just give it a domain name to block (or multiple hosts files/hosts). You can also whitelist individual hosts or entire files by adding them to the end of the list. When a host name is looked up, the query goes to the VPN which looks at the packet and responds with NXDOMAIN (non-existing domain) for hosts that are blocked.

You can find DNS66 here:

F-Droid is the recommended source to install from. DNS66 is licensed under the GNU GPL 3, or (mostly) any later version.

Implementation Notes

DNS66’s core logic is based on another project,  dbrodie/AdBuster, which arguably has the cooler name. I translated that from Kotlin to Java, and cleaned up the implementation a bit:

All work is done in a single thread by using poll() to detect when to read/write stuff. Each DNS request is sent via a new UDP socket, and poll() polls over all UDP sockets, a Device Socket (for the VPN’s tun device) and a pipe (so we can interrupt the poll at any time by closing the pipe).

We literally redirect your DNS servers. Meaning if your DNS server is, all traffic to is routed to the VPN. The VPN only understands DNS traffic, though, so you might have trouble if your DNS server also happens to serve something else. I plan to change that at some point to emulate multiple DNS servers with fake IPs, but this was a first step to get it working with fallback: Android can now transparently fallback to other DNS servers without having to be aware that they are routed via the VPN.

We also need to deal with timing out queries that we received no answer for: DNS66 stores the query into a LinkedHashMap and overrides the removeEldestEntry() method to remove the eldest entry if it is older than 10 seconds or there are more than 1024 pending queries. This means that it only times out up to one request per new request, but it eventually cleans up fine.


Filed under: Android, Uncategorized
Categories: FLOSS Project Planets

Four Kitchens: Launch Announcement: MDedge.com

Planet Drupal - Tue, 2016-10-25 12:16

MDedge is a digital and print publisher providing resources for health industry professionals. […]

Categories: FLOSS Project Planets

myDropWizard.com: Clients don’t want to be billed hourly for site maintenance - even if they say they do!

Planet Drupal - Tue, 2016-10-25 12:11

As Drupal professionals, you and I know that all sites need maintenance and support after they're launched. There's security updates, bugs, minor tweaks and questions to be answered. Clients might not know that, so it's our job to educate them.

We find that it's easiest to discuss this before development starts on their new site, rather than after it's done, but whenever it happens, you'll need to agree to a plan for when things come up.

Most Drupal shops and freelancers default to "hourly as needed". Usually clients will go along with that - or even insist on it!

However, we strongly believe that billing hourly for site maintenance and support is WORSE for clients (and you).

Read more to find why - and learn a better way to do it!

Categories: FLOSS Project Planets

Michal &#268;iha&#345;: New features on Hosted Weblate

Planet Debian - Tue, 2016-10-25 12:00

Today, new version has been deployed on Hosted Weblate. It brings many long requested features and enhancements.

Adding project to watched got way simpler, you can now do it on the project page using watch button:

Another feature which will be liked by project admins is that they can now change project metadata without contacting me. This works for both project and component level:

And adding some fancy things, there is new badge showing status of translations into all languages. This is how it looks for Weblate itself:

As you can see it can get pretty big for projects with many translations, but you get complete picture of the translation status in it.

You can find all these features in upcoming Weblate 2.9 which should be released next week. Complete list of changes in Weblate 2.9 is described in our documentation.

Filed under: Debian English phpMyAdmin SUSE Weblate | 0 comments

Categories: FLOSS Project Planets

Drupalize.Me: The New Drupal 8 Initiatives

Planet Drupal - Tue, 2016-10-25 11:24

You may have heard of a variety of Drupal 8 initiatives during the development cycle leading up to Drupal 8.0.0 being released in 2015. These were officially recognized efforts to get a variety of big changes into Drupal 8, and included projects such as configuration management, Views in core, and multilingual improvements. A lot of work from those initiatives is now part of Drupal 8. Not everything got in though, and as time moves on some priorities for new work will always shift.

Categories: FLOSS Project Planets

My QtCon + Akademy 2016

Planet KDE - Tue, 2016-10-25 10:57

From August 31th to September 10th I was em Berlin attending two amazing conferences: QtCon and Akademy.

QtCon brought together five communities to host their respective conferences at a same time and place, creating one big and diverse conference. Those communities were Qt, KDAB, KDE (celebrating 20th birthday), VLC and FSFE (both celebrating 15th birthday).

Main conference hall of QtCon at bcc

That diversity of themes was a very interesting characteristic of QtCon. I really appreciated see presentations of Qt and KDAB people, and I was surprised about topics related with VLC community. The strong technical aspects of trends like Qt in mobile, Qt in IoT (including autonomous cars), the future of Qt, Qt + Python, contributing to Qt, and more, called my attention during the conference.

On VLC I was surprised with the size of the community. I never imagined VLC had too much developers. In fact, I never imagined VideoLAN is in fact an umbrella of a lot of projects related with multimedia, like codecs, streaming tools, VLC ports to specific devices (including cars through Android Auto), and more. Yes, I really appreciated to find these persons and watch their presentations.

I was waiting for the VLC 3.0 release during QtCon, but unfortunately it did not happen. Of course the team is improving this new release and when it is finished I will have a VLC to use together with my Chromecast, so, keep this great work coneheads!

FSFE presentations were interesting as well. In Brazil there are several talks about political and philosophical aspects of free software in conferences like FISL and Latinoware. In QtCon, FSFE brought this type of presentation in an “European” style: sometimes the presentations looks like more pragmatically in their approaches. Other FSFE presentations talked about the infrastructure and organizational aspects of the foundation, a nice overview to be compared with others groups like ASL.org in Brazil.

Of course, there were a lot of amazing presentations from our gearheads. I highlight the talks about KDE history, Plasma Desktop latest news, Plasma Mobile status, KF5 on Android, the experience of Minuet in mobile world, among others.

The KDE Store announcement was really interesting and I expect it will bring more attention to the KDE ecosystem when software package bundles
(snap/flat/etc) be available in the store.

Other software called my attention was Peruse, a comic book reader. I expect developers can solve the current problems in order to release a mobile version of Peruse, so this software can reach a broad base of users of these platforms.

After the end of QtCon, Akademy had place in TU Berlin, in a very beautiful and comfortable campus. This phase of the conference was full of technical sessions and discussions, hacking, and fun.

I attended  to the Flatpack, Appstream, and Snapcraft BoFs. There were a lot of advanced technical discussions on those themes. Every Akademy I feel very impressed with the advanced level of the technical discussions performed by our hackers in KDE community. Really guys, you rocks!

The Snapcraft BoF was a tutorial about how to use that technology to create crossdistro bundle packages. That was interesting and I would like to test more and give a look in Flatpack in order to select something to create packages for Cantor.

Unfortunately I missed the BoF on Kube. I am very interested in an alternative PIM project for KDE, focused in E-Mail/Contacts/Calendar and more economic in computational resource demand. I am keeping my eyes and expectations on this project.

The others days basically I spent my time working on Cantor and having talk with our worldwide KDE fellows about several topics like KDE Edu, improvements in our Jabber/XMPP infrastructure, KDE 20th years, Plasma in small-size computers (thanks sebas for the Odroid-C1+ device ) WikiToLearn (could be interesting a way to import/export Cantor worksheets to/from WikiToLearn?), and of course, beers and Germany food.

And what about Berlin? It was my second time in the city, and like the previous one I was excited with the multicultural atmosphere, the food (<3 pork <3) and beers. We were in Kreuzberg, a hipster district in the city, so we could visit some bars and expat restaurants there. The QtCon+Akademy had interesting events as well, like the FSFE celebration in c-base and the Akademy daytrip in Peacock Island.

So, I would like to say thank you for KDE e.V. for funding my attendance in the events, thank you Petra for help us with the hostel, and thank your for all the volunteers for work hard and make this Akademy edition a real celebration of KDE community.

Some Brazilians in QtCon/Akademy 2016: KDHelio, Lamarque, Sandro, João, Aracele, Filipe (me)

Categories: FLOSS Project Planets

Mobile IMG 20161025-140532

Planet KDE - Tue, 2016-10-25 10:08

New mobile IMG update,

  • Updated packages
  • Newer libwayland then one available in Ubuntu repositories
  • Experimental activity switcher applet

You can flash using instructions at https://plasma-mobile.org/nexus-5/

Categories: FLOSS Project Planets

qed42.com: New Module - CSSgram recreating Instagram like filters for Drupal 8

Planet Drupal - Tue, 2016-10-25 09:52
New Module - CSSgram recreating Instagram like filters for Drupal 8 Body

CSSgram module supplements Drupal Image styling experience by making Instagram like filters available to your Drupal 8 site images, we do this with help of CSSgram library. 

Beauty of this module is, it simply uses css to beautify your image.

Few CSSGram sample filters applied to an image.


How CSSGram Module works?

CSSGram module uses CSSGram Library for adding filter effects via CSS to the image fields. Module extends Field Formatter Settings to add image filter for that particular field. CSSGram extends field formatter settings and hence these filters can be applied on top of the existing available image formatters and image presets. Allowing you to use your desired image preset along with CSSGram filters.

Using CSSGram
  1. Download and enable CSSGram module (https://www.drupal.org/project/cssgram)

  2. Visit Manage Display of content type and for the desired image field, click on the setting link under format column.

  3. Select Filter option lets us choose from the available image filters. Just select the desired image filter and hit update button.

  1. Save the settings and visit the content display page.

Developer Support

Devs have the option to use these filters anywhere on the site by just attaching the ‘cssgram/cssgram’ library and then applying any of the available css filter class to the wrapper element.

function mymodule_preprocess_field(&$variables) { // Add desired css class. $variables['attributes']['class'] = 'kelvin'; // Attach cssgram library. $variables['#attached']['library'][] = 'cssgram/cssgram'; }


PURUSHOTAM RAI Tue, 10/25/2016 - 19:22
Categories: FLOSS Project Planets

TimOnWeb.com: Add reCaptcha to your Drupal 7 forms programatically

Planet Drupal - Tue, 2016-10-25 08:46

If you want to add Google's reCaptcha (https://www.google.com/recaptcha/intro/index.html) to your Drupal 7 forms programmatically you need to follow these two steps:

1) Install and enable captcha (https://www.drupal.org/project/captcha) and recaptcha (https://www.drupal.org/project/recaptcha) modules. The best ...

Read now

Categories: FLOSS Project Planets

InternetDevels: Some great health care websites built with Drupal

Planet Drupal - Tue, 2016-10-25 08:02

Let’s feel Drupal’s pulse! It looks like the popular site-building platform is doing great. It remains a perfect fit for all kinds of sites.

Read more
Categories: FLOSS Project Planets

qed42.com: Programmatically updating URL aliases using Batch API in Drupal 8

Planet Drupal - Tue, 2016-10-25 05:53
Programmatically updating URL aliases using Batch API in Drupal 8 Body

Drupal has always had excellent support for human-friendly URL’s and SEO in general, from early on we have had the luxury of modules like pathauto offering options to update URL for specific entities, token support and bulk update of URLs. Though bulk update works for most cases, there are some situations where we have to update URLs programmatically, some of the use cases are:

  • Multisite setup -- When your setup has many sites, its inconvenient to use bulk update UI to update aliases for each of your sites. Programmatically updating the aliases is a good choice and can be executed via the update hooks.
  • Conditional Update -- When you wish to update aliases based on certain conditions.


In Drupal 8 Pathauto services.yml file we can see that there is a service named ‘pathauto.generator’ which is what we would need. The class for corresponding to this service, PathautoGenerator provides updateEntityAlias method which is what we would be using here:

public function updateEntityAlias(EntityInterface $entity, $op, array $options = array())


EntityInterface $entity: So, we need to pass the entity (node, taxonomy_term or user) here. String $op: Operation to be performed on the entity (‘update’, ‘insert’ or ‘bulkupdate’). $options: An optional array of additional options.

Now all we need is to loop the entities through this function, these entities may be the user, taxonomy_term or node. We will be using entityQuery and entityTypeManager to load entities, similar to the code below

$entities = [];   // Load All nodes.   $result = \Drupal::entityQuery('node')->execute();   $entity_storage = \Drupal::entityTypeManager()->getStorage('node');   $entities = array_merge($entities, $entity_storage->loadMultiple($result));   // Load All taxonomy terms.   $result = \Drupal::entityQuery('taxonomy_term')->execute();   $entity_storage = \Drupal::entityTypeManager()->getStorage('taxonomy_term');   $entities = array_merge($entities, $entity_storage->loadMultiple($result));   // Load All Users.   $result = \Drupal::entityQuery('user')->execute();   $entity_storage = \Drupal::entityTypeManager()->getStorage('user');   $entities = array_merge($entities, $entity_storage->loadMultiple($result));   // Update URL aliases.   foreach ($entities as $entity) {     \Drupal::service('pathauto.generator')->updateEntityAlias($entity, 'update');   }

This works fine and could be used on a small site but considering the real world scenarios we generally perform this type of one-time operation in hook_update_N() and for larger sites, we may have memory issues, which we can resolve by involving batch API to run the updates in the batch. 

Using Batch API in hook_update_n()

As the update process in itself a batch process, so we can’t just use batch_set() to execute our batch process for URL alias update, we need to break the task into smaller chunks and use the $sandbox variable which is passed by reference to update function to track the progress.  

The whole process could be broken down into 4 steps:

  • Collect number of nodes / entities we would be updating the URL for.
  • Use the $sandbox to store information needed to track progress.
  • Process nodes in groups of a certain number (say 20, in our case ).
  • And finally setting the finished status based on progress.
function mymodule_update_8100(&$sandbox) { $entities = []; $entities['node'] = \Drupal::entityQuery('node')->execute(); $entities['user'] = \Drupal::entityQuery('user')->execute(); $entities['taxonomy_term'] = \Drupal::entityQuery('taxonomy_term')->execute(); $result = []; foreach ($entities as $type => $entity_list) { foreach ($entity_list as $entity_id) { $result[] = [ 'entity_type' => $type, 'id' => $entity_id, ]; } } // Use the sandbox to store the information needed to track progression. if (!isset($sandbox['current'])) { // The count of entities visited so far. $sandbox['current'] = 0; // Total entities that must be visited. $sandbox['max'] = count($result); // A place to store messages during the run. } // Process entities by groups of 20. // When a group is processed, the batch update engine determines // whether it should continue processing in the same request or provide // progress feedback to the user and wait for the next request. $limit = 20; $result = array_slice($result, $sandbox['current'], $limit); foreach ($result as $row) { $entity_storage = \Drupal::entityTypeManager()->getStorage($row['entity_type']); $entity = $entity_storage->load($row['id']); // Update Entity URL alias. \Drupal::service('pathauto.generator')->updateEntityAlias($entity, 'update'); // Update our progress information. $sandbox['current']++; } $sandbox['#finished'] = empty($sandbox['max']) ? 1 : ($sandbox['current'] / $sandbox['max']); if ($sandbox['#finished'] >= 1) { return t('The batch URL Alias update is finished.'); } }

The process of loading the entities will differ in case we are just updating a single entity type say nodes. In that case, we can use loadMultiple to load all the entities at once per single batch operation. That’s a kind of trade-off we have to do according to our requirements. The crucial part is using sandbox variable and splitting the job into chunks for batch processing.

PURUSHOTAM RAI Tue, 10/25/2016 - 15:23
Categories: FLOSS Project Planets
Syndicate content