Feeds

Codementor: How I used Python to find interesting people to follow on Medium

Planet Python - Wed, 2017-10-18 15:16
Medium has a large amount of content, a large number of users, and an almost overwhelming number of posts. When you try to find interesting users to interact with, you’re flooded with visual noise.
Categories: FLOSS Project Planets

Elevated Third: Elevated Third Wins 2017 Acquia Engage Award

Planet Drupal - Wed, 2017-10-18 14:04
Elevated Third Wins 2017 Acquia Engage Award Elevated Third Wins 2017 Acquia Engage Award Ayla Peacock Wed, 10/18/2017 - 12:04

At the 2017 Acquia Engage Conference, our enterprise web project for Pinnacol Assurance won the Financial Services category. As an Acquia Preferred Partner and sponsor of the Acquia Engage conference, we are thrilled to have been ranked beside the world's top Drupal websites. 

Pinnacol.com launched in December of 2016. As Colorado’s leading workers compensation insurer, Pinnacol Assurance needed a Drupal website design that reflected the company’s commitment to first-class service.

Built on Drupal 8, we launched a brand new enterprise content management system, created a one-of-a-kind knowledge hub, and revamped the site’s user experience interface.

“The Pinnacol project was lengthy and complex, we had very specific problems that required creative solutions. Elevated Third developed a high performing, enterprise-level website that continues to exceed our expectations.”

- Hilary Miller, Brand and Marketing Director, Pinnacol Assurance

This year, five of our Drupal websites were finalists in the annual Acquia Engage competition. Partners and customers submitted more than 200 nominations across 17 categories to the program. Our Drupal work ranked in the following categories. 

Powdr Corporation, Digital Experience Finalist

Denver Botanic Gardens, Nonprofit Finalist

Comcast Technology Solutions, Brand Experience Finalist

Firewise USA, Community Finalist

Pinnacol Assurance, Financial Services Winner

“Winning sites set themselves apart in how they grabbed our attention and made us want to learn more,” said CMSWire’s Dom Nicastro, one of the award program jurors. “The first thing I looked for were search and commerce capabilities. It's a Google and Amazon world that we live in. No one comes to a website just for a pretty design, and no one remembers a red call-to-action button versus a blue one. Sites that deliver excellent search and easy transactional experiences won for me.”

Congratulations to all the 2017 Acquia Engage winners!

Categories: FLOSS Project Planets

Zivtech: Common Issues Found During a Drupal Site Audit

Planet Drupal - Wed, 2017-10-18 12:43

Over the years, Zivtech has worked on many different types of existing Drupal websites and web applications. These projects have ranged from sites that were built with Drupal’s best practices to those built by developers with little to no Drupal experience. At Zivtech, we typically call a website that follows little to none of Drupal’s best practices a “lemon.” Our CTO, Jody Hamilton, did a great two part blog series called Lemon: Drupal Diseases and Cures if you would like to know more about what a Drupal lemon is.

If your site fits into the category of a lemon, it likely requires too much work to fix and should probably be rebuilt. In many cases though, our developers find that we can “rescue” the site to get it back into a secure and maintainable state after a site audit. We perform a site audit to identify the issues that can be fixed, and then provide an informative report about what problems have been found and how we can resolve them.

Our extensive experience with site audits has helped us identify common mistakes that Drupal sites are often guilty of. In this post we’ll outline the common mistakes and configuration issues that we find on most lemons. Some of these issues are even common on sites that have mostly followed Drupal’s best practices.

Read more
Categories: FLOSS Project Planets

Stack Abuse: Python Tutorial for Absolute Beginners

Planet Python - Wed, 2017-10-18 11:43

Python is one of the most widely used languages out there. Be it web development, machine learning and AI, or even micro-controller programming, Python has found its place just about everywhere.

This article provides a brief introduction to Python for beginners to the language. The article is aimed at absolute beginners with no previous Python experience, although some previous programming knowledge will help, but is not necessarily required.

I've found that the best way to learn is to try to understand the theory and then implement the example on your own. Remember, you will not get better at programming unless you practice it!

The article is divided into following sections:

Why Learn Python

The question arises here that why you should learn Python. There are lots of other programming languages; you might have even learned some of them. Then why Python, what is so special about it? There are various reasons to learn Python, most important of which have been enlisted below.

  • Easy to Learn

    Python is considered one of the most beginner-friendly languages. The syntax of Python is the simplest of all. You don't have to learn complex variable types, use of brackets for grouping code blocks and so on. Python is built upon the fundamental principle of beginner-friendliness.

  • Highly In-Demand

    According to a recent survey by indeed.com, Python developers are the second highest paid developers in USA. The huge job potential of Python can be estimated by the fact that in 2014 the average hiring rate for programmers decreased by 5% but Python developers still saw an increase of 8.7%.

  • Ideal for Web development

    Python is lightning fast when compared with other web development languages such as PHP and ASP.NET. Also, Python has myriad of amazing frameworks such as Django, Flask, and Pylons, which makes web development even simpler. Websites like Instagram, Pinterest and The Guardian are all based on the popular Django framework.

  • Used Heavily for Machine learning and AI

    Python is the most widely used language for machine learning and artificial intelligence operations. Python libraries such as TensorFlow and scikit-learn makes AI tasks much simpler when compared to MATLAB or R, which previously was the most widely used environment for data science and AI tasks.

  • Works with Raspberry Pi

    Python is the most popular programming language for the Raspberry Pi, which is a pocket size microcomputer used in a wide range of applications such as robots, gaming consoles, toys. In short, learn Python if you want to build things with the Raspberry Pi.

  • Corporate Darling

    It would not be an exaggeration if we say that Python is the darling of all all the big corporate companies such as google, yahoo, NASA, Disney, IBM etc. These companies have incorporated Python at the core of many of its applications.

  • Large Community

    Python has one of the largest programming communities online and it continues to grow. Python has the fifth largest Stack Overflow community, and third largest meet-up community. And most importantly, it is the 4th most used language at GitHub, which means there is tons of existing code to learn from.

Installation and Setup

Though there are several ways to install Python for Windows, but for the sake of this article we will use Anaconda. It is undoubtedly the most widely used Python environment at the moment. To download Anaconda, go to this link:

https://www.anaconda.com/download/

Scroll down a bit and you should see the download options. Select, Python 3.6 as shown in the following screenshot:

This will download an Anaconda installer to your computer. Open the installer and you will see the following options:

Follow these steps for installation

  • Click the "Next" button. Terms and Condition will appear, you can read if you have enough time but you can click "I Agree" anyways.
  • In the next window select the type of installation you want. If you are absolute beginner to Python I would recommend selecting, "Just me" option.
  • Next, select the installation folder (Default is best).
  • Advance options dialogue box will appear, keep the first option unchecked and the second checked and click "Install". This is shown in the following screenshot.

Now sit back and have some coffee, the installation might take some time.

Once the installation is complete, you will see the message:

Click "Next" and then "Finish" button on the subsequent dialogue box to complete the installation.

Running your First Program

Although you can run Python programs via command line as well, it is typically better for beginners to use a text editor. Luckily, with the installation of Anaconda, you get the Jupyter Notebook installed as well. The "Jupyter Notebook" is a cloud based application that allows users to create, share, and manage their documents. We will use Jupyter to write our Python code in this article.

To open Jupyter, you can go to Start Menu and find the "Jupyter Notebook" application. You can also search for it in Applications. This is shown in the following:

Open the "Jupyter Notebook" application. It will then be opened in your default browser. For compatibility, I would recommend that you use Google Chrome as your default browser, but other browser types like Firefox would work as well.

When the application opens in your browser, you will see the following page:

On the right hand side of the page, you will see an option "New". Click that buttonand a dropdown list will appear. Select, "Python 3" from the dropdown list. This will open a brand new notebook for you, which looks like this:

Here you can easily write, save, and share your Python code.

Let's test and make sure everything is working fine. To do this, we'll create a simple program that prints a string to the screen.

Enter the following code in the text field in your Jupyter notebook (shown in the screenshot above):

print("Welcome to Python!")

The print does exactly what it sounds like, it simply prints some text to the screen. The text you want to display is entered within the double quotations inside the parenthesis that follow the print keyword.

To run code in "Jupyter Notebook" just press "Ctrl + Enter". The output of the above code should look like the following:

And there you have it, we have successfully executed our first Python program! In the following sections, we'll continue to use Jupyter to teach and discuss some core Python features, starting with variables.

Python Variables

Simply put, variables are memory locations that store some data. You can use variables to store a value, whether it be a number, text, or a boolean (true/false) value. When you need to use that value again later in your code, you can simply use the variable that holds that value. You can almost think of them as simple containers that store things for you for later use.

It is important to mention here that unlike Java, C++ and C#, Python is not a strongly typed language. This means that you do not need to specify the type of variable according to the value it holds. Python implicitly decodes the variable type at runtime depending upon the type of data stored in it. For instance you don't need to specify int n = 10 to define an integer variable named "n". In Python we simply write n = 10 and the type of variable "n" will be implicitly understood at runtime.

There are five different core data types in Python:

  • Numbers
  • Strings
  • List
  • Tuples
  • Dictionaries

In this section we will only take a look at numbers and strings. Lists, tuples, and dictionaries will be explained further in their own respective section later in this article.

Numbers

The number type of variables store numeric data. Take a look at the following simple example:

num1 = 2 num2 = 4 result = num1 + num2 print(result)

Here in the above example we have two numeric variables, num1 and num2, with both containing some numeric data. There is a third number type variable, result, which contains the result of the addition of the values stored in num1 and num2 variables. Finally, on the last line the result variable is printed to the screen.

The output will be as follows:

There are four different number data types in Python:

  • Integers, such as real whole-valued numbers: 10
  • Long integers, which have "L" at the end for values: 1024658L
    • These can also be used in hexadecimal and octal form
  • Floating point data, which are numbers expressed in decimals: 3.14159
  • Complex data, which is used to represent complex number types: 2 + 3j
Strings

Strings are used to store text data in Python. Take a look at the following example:

fname = "Adam" sname = " Grey" fullname = fname + sname print(fullname)

In the above example we have two string variables: fname and sname. These store the first name and surname of some person. To combine these two strings we can use "+" operator in Python. Here we are joining the fname and sname variables and store the resultant string in the fullname variable. Then we print the fullname variable to the screen.

The output is as follows:

There are hundreds of strings operations in Python, we will have a dedicated article on these functions in future.

Operators in Python

Operators in programming are the constructs that allow you to manipulate an operand to perform a specific function. They are very similar to real life operators, such as arithmetic operators e.g addition, subtraction, greater than, less than, and AND/OR operators, etc.

There are seven types of operators in Python:

  • Arithmetic Operators
  • Logical Operators
  • Assignment Operators
  • Comparison Operators
  • Bitwise Operators
  • Identity Operators
  • Member Operators

In this article we'll keep it simple and study only the first four operators. The other operators are beyond the scope of this article.

Arithmetic Operators

Arithmetic operators perform mathematical operations such as addition, subtraction, multiplication, division, and exponential functions on the operands. The detail of arithmetic functions have been given in the following table:

Suppose the variables n1 and n2 have values of 4 and 2, respectively.

Operator Functionality Example Addition (+) Adds two or more operands n1 + n2 = 6 Subtraction (-) Subtracts second operand from the first n1 - n2 = 2 Multiplication (*) Multiply two or more operands n1 * n2 = 8 Division (/) Divide the first operand by second n1 / n2 = 2 Modulus (%) Finds remainder when the first operand is divided by second n1 % n2 = 0 Exponent (**) Takes the power of first operand to the second n1 ** n2 = 16

You may recall seeing an example of the arithmetic addition operator earlier in the Number data variable section. In Python, addition operators can apply to any kind of number, and even strings.

Logical Operators

The logical operators, which help you perform simple Boolean algebra, supported by Python are as follows:

Suppose o1 and o2 have values True and False, respectively.

Operator Functionality Example AND Returns true of the all the conditions are true (o1 and o2) is false OR Returns true if any of the condition is true (o1 or o2) is true NOT Return the reverse of the actual logical state Not(o1) is false

The following code helps explain the above operators with an example:

o1 = True o2 = False r1 = (o1 and o2) print(r1) r2 = (o1 or o2) print(r2) r3 = not(o1) print(r3)

The output of the above code is:

False True False Assignment Operators

Assignment operators allow you to "give" a value to variables, which may be the result of an operation. The following table contains some of the most widely used assignment operators in Python:

Operator Functionality Example Assign Assign the value on the right to the variable on the left r = n1 + n2 assigns value of n1 + n2 into r Add and Assign Adds both the operands and assign the result to the variable on the left one n1 += n2 is the same as n1 = n1 + n2 Subtract and Assign Subtracts right operand from the left and assign the result to the left n1 -= n2 means n1 = n1 - n2 Multiply and Assign Multiplies both the operands and assign the result to the left one n1 *= n2 means n1 = n1 * n2 Divide and Assign Divides both the operands and assign the result to the left one n1 /= n2 means n1 = n1 / n2 Modulus and Assign Take modulus by dividing left operand by the right and assign the result to the left one n1 %= n2 means n1 = n1 % n2 Exponent and Assign Take exponent of right operand to the left and assign the value to left n1 **= n2 means n1 = n1 ** n2

Take a look at the following example to see some of the assignment operators in action:

n1 = 4 n2 = 2 n1 += n2 print(n1) n1 = 4 n1 -= n2 print(n1) n1 = 4 n1 *= n2 print(n1) n1 = 4 n1 /= n2 print(n1)

The output of the above code will be:

6 2 8 2.0

Notice how in the last operation we get a floating point number as our result, whereas we get integer numberes in all of the prevoius operations. This is because this is the only mathematical operation in our example that could turn two integer numbers in to a floating point number.

Comparison Operators

Comparison operators are used to compare two or more operands. Python supports the following comparison operators:

Suppose n1 is 10 and n2 is 5 in the following table.

Operator Functionality Example == Returns True if the two operands are equal to each other (n1 == n2) is not true != Returns true of two operands are not equal (n1 != n2) is true > Returns true if left operand is greater than right operand (n1 > n2) is true < Returns true if left operand is smaller than right operand (n1 < n2) is not true >= Returns true if left operand is equal to or greater than the right operand (n1 >= n2) is true <= Returns true if left operand is equal to or smaller than the right operand (n1 =< n2) is not true

Consider the following simple example of comparison operator:

n1 = 10 n2 = 5 print(n1 == n2) print(n1 != n2) print(n1 > n2) print(n1 < n2) print(n1 >= n2) print(n1 <= n2)

The output of the above code is:

False True True False True False The Complete Python Bootcamp Want to learn more about Python than what is covered in this article? With this course you'll get over 100 lectures and more than 10 hours of video. Start from the basics and go all the way to creating your own applications and games! Conditional Statements

Conditional statements are used to select the code block that you want to execute based upon a certain condition. Suppose in a hospital management system, you want to implement a check that the patient with age over 65 can receive priority treatment while the others cannot, you can do so with conditional statements.

There are four types of conditional statements:

  • "if" statements
  • "if/else" statements
  • "if/elif" statement
  • Nested "if/else" statements

Basically, the second and third types are just extensions of the first statement type.

If Statement

The "if statement" is the simplest of all the statements. If the given condition resolves to true (like 1 < 10), then the code block that follows the "if statement" is executed. If the condition returns false (like 1 > 10), then the code is not executed.

Take a look at the following example.

age = 67 if age >= 65: print("You are eligible for priority treatment.") print("Thank you for your visit")

Pay close attention to the syntax of conditional statements. In most of the other programming languages, the code block that is to be executed if the "if" condition returns true is enclosed inside brackets. Here in Python you have to use colon after the "if" condition and then you have to indent the code that you want to execute if the condition returns true.

Python is widely considered to be a much cleaner language than many others because of the absence of brackets. Indentation is used instead to specify scope, which has its own pros and cons.

In the above example we have an age variable with value 67. We check if age is greater than 65, and if this condition returns true then we print a message telling the user that he/she is eligible for priority treatment. Notice that this message is indented, which tells us it is the code to be executed following a true condition. Finally, we simply print the thank you message on the screen. The output of this code will be:

You are eligible for priority treatment. Thank you for your visit

Now let's set the value of the age variable to 55 and see the difference.

age = 55 if age >=65: print("You are eligible for priority treatement.") print("Thank you for your visit")

The output of the above looks like this:

Thank you for your visit

Notice that this time the condition did not return true, hence the statement telling the patient that he is eligible for priority treatment is not printed to the screen. Only greetings have appeared since they were not inside (indented) the body of the "if" statement.

If/Else Statement

The "if/else" statement is used to specify the alternative path of execution in case the "if" statement returns false. Take a look at the following example:

age = 55 if age >=65: print("You are eligible for priority treatment.") else: print("You are eligible for normal treatment") print("Thank you for your visit")

Here the code block followed by the "else" statement will be executed since the age variable is 55 and the "if" condition will return false. Hence, the "else" statement will be executed instead. The output will be as follows:

You are eligible for normal treatment Thank you for your visit If/Elif Statment

The "if/elif" statement is used to implement multiple conditions. Take a look at the following example:

age = 10 if age >= 65: print("You are eligible for priority treatment.") elif age > 18 and age < 65: print("You are eligible for normal treatment") elif age < 18: print("You are eligible for juvenile treatment") print("Thank you for your visit")

In the above code we have implemented three conditions. If age is greater than 65, if age is between 65 and 18, and if the age is less than 18. Based on the value of the age, different print statement will be executed. Here since the age is 10, the second conditional returns true and you will see the following output:

You are eligible for juvenile treatment Thank you for your visit

If none of the conditionals were to return true then none of the print() statements would have executed. This differs from the "if/else" example where either "if" is executed or "else" is executed. In the case of "if/elif", this isn't necessarily the case. However, you can add a normal "else" statement at the end that gets executed if none of the conditions above it return true.

Using this method I just described, we could re-write the previous example to look like this:

age = 10 if age >= 65: print("You are eligible for priority treatment.") elif age > 18 and age < 65: print("You are eligible for normal treatment") else: print("You are eligible for juvenile treatment") print("Thank you for your visit")

This code would result in the same output as the previous example.

Nested If Else Statement

Nested "if/else" statements are used to implement nested conditions (i.e. conditions within another condition). Consider the following example:

age = 67 insurance = "yes" if age >= 65: print("You are eligible for priority treatment.") if insurance == "yes": print("The insurance company will pay for you.") else: print("You have to pay in advance.") else: print("You are eligble for normal treatment") print("Thank you for your visit")

Here we have an outer condition that if age is greater than or equal to 65, then check that if patient has insurance or not. If the patient has insurance, the insurance company will pay the bill later, otherwise the patient has to pay in advance.

Loops

Iteration statements, or more commonly known as loops, are used to repeatedly execute a piece of code multiple times. Consider if you have to print names of 100 persons on the screen. You will either have to write 100 print statements or you will have to use hundreds of escape characters in one print statements. If you have to perform this task repeatedly you have to write hundreds of thousands of tedious lines of code. A better way is to make use of loops.

There are two main types of loops in Python:

  • For loop
  • While Loop

Keep in mind that you can nest loops just like we did with the conditional statements, but we won't go in to that here.

The For Loop

The "for loop" is used to iterate over a collection of elements. The loop keeps executing until all the elements in the collection have been traversed. Take a look at the simple example of for loop:

nums = [1, 2, 4, 5, 6, 7, 8, 9, 10] for n in nums: print(5 * n)

The above example simply prints the product of each item in nums and 5. Here we have a list nums which contains integers from 1 to 10. Don't worry, we will study lists in detail in a later section. For now, just consider it as a collection of elements, which in this case are numbers.

Pay close attention to the code above. It follows following syntax:

for [temp_var] in [collection]: [statements]

In the first iteration of the "for loop" the 1 is stored in the temporary variable n. This 1 is multiplied by 5 and the result is printed on the screen. In the second iteration the second element from the nums collection (i.e. 2) is stored in the n variable and 2 is multiplied by 5. These iterations continue until all the elements in the nums collection have been traversed. After the last element (10) is encountered, the loop stops and code execution moves past the "for loop".

The output of the above code is:

5 10 20 25 30 35 40 45 50 The While Loop

The "while loop" is different from the "for loop" in that it keeps executing while a certain condition keeps returning true. After each iteration of the while loop, the condition is re-evaluated. When the condition finally returns false, the while loop stops executing and exits.

Take a look at the following example:

x = 50 while x > 0: print(x) x = x - 5

Here the loop will keep executing until the value of x becomes negative. The x variable has initially value of 50 and during each iteration we decrement it by 5. So, after 10 iterations the value will become negative and the loop will then stop executing.

The output will look like this:

50 45 40 35 30 25 20 15 10 5

While loops are good for times when you don't already know how many iterations you need. For loops iterate a set number of times, whereas while loops can iterate an unknown number of times, or even an infinite number of times.

Functions in Python

Functions in programming are constructs that perform specific tasks. Functions come handy in scenarios when you have to perform a task multiple times throughout your code. Instead of re-writing the same functionality again and again, instead you can create a function that performs that task and then call that function wherever and whenever you want.

Notice that there is a difference between doing a task repeatedly and doing a task multiple times. Loops are used where you have to perform a task repeatedly in sequence. Functions, on the other hand, are used when you have to perform the same task at different places throughout your code.

Consider a scenario where you have to print a long statement to screen at different times. Instead, write a function that prints the statement you want and then call the function wherever you want to print the statement.

Take a look at the following example:

def displayWelcome(): print("Welcome to Python. This article explains the basics of Python for absolute beginners!") return; displayWelcome() print("Do something here") displayWelcome() print("Do some other stuff here")

There are two things I'd like to point out in this code: the function definition and the function calls.

Function definition refers to defining the task performed by the function. To define a function you have to use keyword def followed by the name of the function, which is displayWelcome in the above example. You can use any function name, but to use semantic function. The function name is followed by opening and closing parenthesis. The parenthesis are used to define parameters or any default input values, which we will see this in next example. After the parenthesis you have to use colon and on the next line the body of the function is defined. A function usually ends with a return statement, but it is not required if a value is not being returned.

In the second part of our example code you'll see the function call. To call a function you simply have to write the function name followed by pair of parenthesis. If a function accepts parameters, you have to pass them inside parenthesis.

The output of the above code will be:

Welcome to Python. This article explains the basics of Python for absolute beginners Do something here Welcome to Python. This article explains the basics of Python for absolute beginners Do some other stuff here

You can see that our long string was printed twice. Once before the "Do something here" statement, and once after it, which matches the order of our function calls within the code.

You can imagine how important this is to programming. What if we needed to perform a more complex task like downloading a file or performing a complex calculation? It would be wasteful to write out the full code multiple times, which is where functions come in to play.

Functions with Parameters

Now let's see how to pass parameters to a function. A parameter is just a variable that is given to the function from the caller.

Let's write a function that adds two numbers passed to it as parameters in the parenthesis:

def addNumbers(n1, n2): r = n1 + n2 return r; result = addNumbers(10, 20) print(result) result = addNumbers(40, 60) print(result) result = addNumbers(15, 25) print(result)

In the above code we have the addNumbers function, which accepts two values from the function call. The values are stored in the n1 and n2 variables. Inside the function these values are added and stored in the r variable. The value in the r variable is then returned to the caller of the function.

In the first call to addNumbers we pass two values, 10 and 20. Note that the order of parameters matter. The first value in the function call is stored in the first parameter in the function, and the second value is stored in the second parameter. Therefore 10 will be stored in n1 and 20 will be stored in n2. We then display the result of the function via the print statement. This function is called a total of three times, each time with different parameter values.

The result of the above code will be:

30 100 40

You can see that every time the function is called, our result variable contains the addition of the two numbers passed.

Lists, Tuples, and Dictionaries

Lists, tuples, and dictionaries are three of the most commonly used data structures in programming. Though all of them store a collection of data, the main difference lies in the following:

  • How you place data in the data structure
  • How the data is stored within the structure
  • How data is accessed from the data structure

In the next few sections you'll see some of these properties for each data structure.

Lists

Lists are used to store a collection of items of varying data types. The elements are stored inside square brackets where each element is separated from each other with a comma.

Let's see how to create a simple list:

randomlist = ['apple', 'banana', True, 10, 'Mango']

You can see we have stored strings, a number, and a Boolean in this list. In Python (unlike other strongly typed languages), a list can store any type of data in a single list, as shown above. More commonly, however, lists tend to store many different values of the same data type.

Accessing List Elements

To access an element in a list simply write the name of the list variable followed by pair of square brackets. Inside the brackets specify the index number of the element you want to access. It is important to note that lists in Python (and many other programming languages), list indexes start at 0. This means the first element in every list is at position 0, and the last element is at position n-1, where n is the length of the list. This is called zero-based indexing.

Take a look at this code:

print(randomlist[0]) print(randomlist[4])

Here we are accessing the first and fifth element of the randomlist list. The output will be:

apple Mango

You may have also noticed that the elements in the list remain in the order in which they are stored. They will remain in the same order unless they are explicitly moved or they are removed.

Assigning New List Elements

To assign a value to an existing list position, you must specify the index of the position you want to assign the value to and then use the assignment operator (=) to actually assign the value.

See the code below:

# Define the list randomlist = ['apple', 'banana', True, '10', 'Mango'] # Print the current value at index 0 print(randomlist[0]) # Assign a new value at index 0 randomlist[0] = 'Peach' # Print the updated value print(randomlist[0])

Here we have updated the first element of the list. We displayed the value of the element before and after the update to show the change.

Adding List Elements

In the last sub-section we showed how to assign a value to a list, but this only applies if an item already exists at that position. What if we wnat to expand the size of the list and add a new item without getting rid of any of our previous items? We do this by using the append() function.

randomlist = ['apple', 'banana', True, '10', 'Mango'] print(randomlist) # Add a new element randomlist.append(0) print(randomlist)

When running this code you will notice that the value 0 is shown at the end of the list after calling the append function. Our list now has a total of 6 elements in it, including our new value.

Deleting List Elements

To remove an element, we simply use the del keyword. Take a look at the following example to see how it is used:

randomlist = ['apple', 'banana', True, '10', 'Mango'] print(randomlist) # Remove the second element del randomlist[1] print(randomlist)

Here we deleted the second element of the randomlist list. We use the print statement to show the list before and after deleting the element. The output will be as follows:

['apple', 'banana', True, '10', 'Mango'] ['apple', True, '10', 'Mango'] Tuples

Tuples are similar to list in that they store elements of varying data types. The main distinction between tuples and lists is that tuples are immutable. This means that once you have created a tuple you cannot update the value of any element in the tuple, nor can you delete an element.

In terms of syntax, tuples differ from lists in that they use parenthasis, whereas lists use square brackets. Even with all of these differences, tuples are still very similar to lists. Elements are accessed the same, and element order is preserved, just like lists.

Here is how you can create a tuple:

randomtuple = ('apple', 'banana', True, '10', 'Mango') Accessing Tuple Elements

Tuple elements can be accessed in same way as lists:

randomtuple = ('apple', 'banana', True, '10', 'Mango') print(randomtuple[1]) print(randomtuple[4])

In the above script we are accessing the second and fifth element of the tuple. As expected, this would result in the following output:

banana Mango Assigning Values to Tuple Elements

As discussed earlier, it is not possible to assign new values to already declared tuple elements. So you cannot do something like this:

randomtuple[1] = 10 # This operation is not allowed

Attempting an assignment like this results in the following error being raised:

TypeError: 'tuple' object does not support item assignment Deleting a Tuple Element

You cannot delete an individual tuple element. Attempting to do so would result in a raised error, just like we showed when you try to re-assign an element:

TypeError: 'tuple' object doesn't support item deletion

However you can delete a tuple itself using "del" function as shown in the following example:

randomtuple = ('apple', 'banana', True, '10', 'Mango') print(randomtuple) del randomtuple print(randomtuple)

If you try to access a deleted tuple, as in the second print statement above, you will receive the following error message:

NameError: name 'randomtuple' is not defined Dictionaries

Like lists and tuples, dictionary data structures store a collection of elements. However, they differ quite a bit from tuples and lists because they are key-value stores. This means that you give each value a key (most commonly a string or integer) that can be used to access the element at a later time. When you have a large amount of data, this is more efficient for accessing data than traversing an entire list to find your element.

When you create a dictionary, each key-value pair is separated from the other by a comma, and all of the elements are stored inside curly brackets. See the following code:

randomdict = {'Make': 'Honda', 'Model': 'Civic', 'Year': 2010, 'Color': 'Black'}

Dictionaries are very useful when you have a lot of information about a particular thing, like the car example we showed above. They're also useful when you need to access random elements in the collection and don't want to traverse a huge list to access them.

Accessing Dictionary Elements

Dictionary elements are accessed using their keys. For instance if you want to access the first element, you will have to use its key, which in this case is 'Make'. Take a look at the following example to see the syntax:

randomdict = {'Make': 'Honda', 'Model': 'Civic', 'Year': 2010, 'Color': 'Black'} print(randomdict['Make']) print(randomdict['Model'])

Here we are accessing the first and second elements of the randomdict dictionary via their keys. The output will look like this:

Honda Civic

Because dictionary elements are accessed using their keys, the elements are not ordered in the data structure, and it is not as straight-forward to iterate over like lists are.

Assigning Values to Dictionary Elements

To assign value to already existing dictionary element you first have to access the element and then assign a new value to it. The following example shows this:

randomdict = {'Make': 'Honda', 'Model': 'Civic', 'Year': 2010, 'Color': 'Black'} print(randomdict['Make']) randomdict['Make'] = 'Audi' print(randomdict['Make'])

The output will have this:

Honda Audi Deleting Dictionary Elements

There are three different ways to delete elements in dictionaries: You can delete individual elements, you can delete all the elements, or you can delete the entire dictionary itself. The following example shows all of these three ways:

randomdict = {'Make': 'Honda', 'Model': 'Civic', 'Year': 2010, 'Color': 'Black'} # Displaying complete dictionary print(randomdict) # Deleting one element del randomdict['Make'] print(randomdict) # Clearing whole dictionary randomdict.clear() print(randomdict) # Deleting dictionary itself del randomdict print(randomdict)

Here we are displaying the dictionary after performing each of the three delete operations. Don't worry about the "#" and proceeding text in the code - these are there to make comments about the code. Comments are not executed, they just provide information about the code, and are purely optional.

The output of the above code will be:

{'Color': 'Black', 'Make': 'Honda', 'Model': 'Civic', 'Year': 2010} {'Color': 'Black', 'Model': 'Civic', 'Year': 2010} {} Traceback (most recent call last): File "dict_test.py", line 16, in <module> print(randomdict) NameError: name 'randomdict' is not defined

Notice that since we deleted the dictionary at the end, therefore an error is thrown indicating that randomdict is not defined.

Example Application

Now that we've gone through many of the most basic concepts in Python, let's put it to good use and create an simple appplication using what we learned.

Let's say you have so many cars that you just can't keep track of them all, so we'll create an application to do it for you. It'll work by continually asking you if you want to add cars to your inventory, and if you do, then it will ask for the details of the car. If you don't, the application will print out the details of all of your cars and exit.

Here is the full code, which we'll explain in detail in the rest of this section:

cars = [] add_inventory = raw_input('Add inventory? [y/n] ') while add_inventory == 'y': # Get car data from user make = raw_input('Make: ') model = raw_input('Model: ') year = raw_input('Year: ') miles = raw_input('Miles: ') # Create car dictionary object and save it to list car = {'Make': make, 'Model': model, 'Year': year, 'Miles': miles} cars.append(car) # Ask user if we should keep going add_inventory = raw_input('Add inventory? [y/n] ') print('') print('Here are your cars:') # Display all of our cars for c in cars: print('Make: ' + c['Make']) print('Model: ' + c['Model']) print('Year: ' + c['Year']) print('Miles: ' + c['Miles']) print('')

In the first line of our code we create a list that will hold the details of all of our cars. Each element in the list will be a dictionary item, which will contain details like "Make", "Model", etc.

The second line of code we use a built-in Python function called raw_input(), which displays the given text to the user via the command line and then waits for the response. Any text that is entered by the user is then saved in the add_inventory variable.

We then check if the user wanted to add inventory by checking for a "y" character. If the user does want to add inventory, then we use the raw_input() function again to gather information about the car. Once we have everything we need, we create a car variable that stores a dictionary with all of our car data. This dictionary object is then saved in our car list using the append() method, which you may recall adds our element to the end of the list.

Using a "while-loop", we continually check to see if the user wants to add more cars to their inventory. This could go on for as long as the user keeps entering "y" in the "Add inventory?" prompt, which is exactly what "while-loops" are good for.

When the user finally enters "n" (or any character that isn't "y"), we will print out a full list of their inventory for them. This is done using a "for-loop". For each item in the list, we store the current item in the temporary c variable and retrieve all of the relevant car data using its keys, which we then print out to the screen using string concatenation (or "addition"). This adds the two strings together to become one before getting printed to the screen.

Running this code via the command line may look something like this:

$ python cars.py Add inventory? [y/n] y Make: Porsche Model: 911 Turbo Year: 2017 Miles: 2000 Add inventory? [y/n] y Make: Ferrari Model: 488 GTB Year: 2016 Miles: 12000 Add inventory? [y/n] y Make: Lamborghini Model: Aventador Year: 2017 Miles: 8000 Add inventory? [y/n] n Here are your cars: Make: Porsche Model: 911 Turbo Year: 2017 Miles: 2000 Make: Ferrari Model: 488 GTB Year: 2016 Miles: 12000 Make: Lamborghini Model: Aventador Year: 2017 Miles: 8000 What's next?

This article provides a very basic introduction to the Python programming language. We have touched on only the most fundamental concepts, including variables, operators, conditional statements, loops, and more.

An entire article could be dedicated to each of these topics, so I'd suggest finding more resources on each. To learn more, personally I'd recommend taking a course like Complete Python Bootcamp: Go from zero to hero in Python, which will guide you through all of the most important concepts in greater detail.

Another great one is the Complete Python Masterclass, which goes even further in to things like object-oriented programming and even databases.

Once you find your feet in the simple Python concepts, move on to more advanced topics like object-oriented Python. Most of the advanced programming applications now-a-days are based on object oriented principles. As explained in the beginning, Python is being widely used for web development, machine learning, data science, and micro-controllers as well, so try out a little of everything and see which niche is most interesting to you.

What do you think of Python so far? What do you plan on using it for? Let us know in the comments!

Categories: FLOSS Project Planets

Calvin Spealman: On Pruning Your Passions

Planet Python - Wed, 2017-10-18 11:12
We live in a hobby-rich world. There is no shortage of pastimes to grow a passion for. There is a shortage of one thing: time to indulge those passions. If you're someone who pours your heart into that one thing that makes your life worthwhile, that's a great deal. But, what if you've got no shortage of interests that draw your attention and you realize you will never have the time for all of them?

If I look at all the things I'd love to do with my life as a rose bush I'm tending, I realize that careful pruning is essential for the best outcome. This is a hard lesson to learn, because it can mean cutting beautiful flowers and watching the petals fall to the ground to wither. It has to be done.

I have a full time job that takes a lot of my mental energy. I have a wife and a son and family time is very important in my house. I try to read more, and I want to keep up with new developments in my career, and I'm trying to make time for simple, intentional relaxing to lower my anxiety and stress. That doesn't leave a lot of room to pursue any of these hobbies.

I used to play the guitar, if only a bit.

I've always had my eye on becoming a writer.

Software development began as a passion hobby, and now that it is a carrier I still feel that draw to it outside of work.

A lot of my life was spent under the assumption that I would end up in some career as an artist, and I was even on a trajectory towards art school in my teens.

But there aren't enough days in the year, or hours in any of those days, to scratch 100% of those itches.

So, I'm committing to saying "No" to myself more often. When I'm looking for a small app or tool and can't find just the right thing, I'm going to say "No" to building my own, instead of making the best option work. When NaNoWriMo rolls around next year, I'm not going to cause myself anxiety over the "Will I? Won't I?" leading up, and I'm going to admit that it just doesn't work for me. When I end my work day, I'm going to leave the web development at work.

I will be saying "No" to myself on all these interests so I can direct my "Yes" whole heartedly to one: my blossoming foray into game development. And this is a really deliberate choice! Game development is what got me into computers and into programming. But, its also something multi-faceted in a way that few other pursuits are. By throwing myself fully into my game projects, I'll be able to spend time created art, to code outside of work and learn new techniques and paradigms, and to tell stories.

I'm putting down a lot of interests, and shelving a lot of personal projects. I have dozens of bits of code that'll only collect dust from now on, even though I think of them often and constantly feel the pull to hack on them in the evening or weekends. But, I have convinced myself this is for the best. I'm working on making 2017 a big year for me, and I can't do that when I'm pulled in a thousand directions.

Learning to give up just may be the ticket to finally succeeded.


Categories: FLOSS Project Planets

PyPy Development

Planet Python - Wed, 2017-10-18 10:39
(Cape of) Good Hope for PyPy
Hello from the other side of the world (for most of you)!

With the excuse of coming to PyCon ZA during the last two weeks Armin, Ronan, Antonio and sometimes Maciek had a very nice and productive sprint in Cape Town, as pictures show :). We would like to say a big thank you to Kiwi.com, which sponsored part of the travel costs via its awesome Sourcelift program to help Open Source projects.

Armin, Anto and Ronan at Cape Point
Armin, Ronan and Anto spent most of the time hacking at cpyext, our CPython C-API compatibility layer: during the last years, the focus was to make it working and compatible with CPython, in order to run existing libraries such as numpy and pandas. However, we never paid too much attention to performance, so the net result is that with the latest released version of PyPy, C extensions generally work but their speed ranges from "slow" to "horribly slow".

For example, these very simple microbenchmarks measure the speed of calling (empty) C functions, i.e. the time you spend to "cross the border" between RPython and C. These are the results on CPython, on PyPy 5.8, and on our newest in-progress version:

$ python bench.py # CPython noargs : 0.41 secs onearg(None): 0.44 secs onearg(i) : 0.44 secs varargs : 0.58 secs
$ pypy-5.8 bench.py # PyPy 5.8 noargs : 1.01 secs onearg(None): 1.31 secs onearg(i) : 2.57 secs varargs : 2.79 secs
$ pypy bench.py # cpyext-refactor-methodobject branch noargs : 0.17 secs onearg(None): 0.21 secs onearg(i) : 0.22 secs varargs : 0.47 secs
So yes: before the sprint, we were ~2-6x slower than CPython. Now, we are faster than it! To reach this result, we did various improvements, such as:
  1. teach the JIT how to look (a bit) inside the cpyext module;
  2. write specialized code for calling METH_NOARGS, METH_O and METH_VARARGS functions; previously, we always used a very general and slow logic;
  3. implement freelists to allocate the cpyext versions of int and tuple objects, as CPython does;
  4. the cpyext-avoid-roundtrip branch: crossing the RPython/C border is slowish, but the real problem was (and still is for many cases) we often cross it many times for no good reason. So, depending on the actual API call, you might end up in the C land, which calls back into the RPython land, which goes to C, etc. etc. (ad libitum).
The branch tries to fix such nonsense: so far, we fixed only some cases, which are enough to speed up the benchmarks shown above. But most importantly, we now have a clear path and an actual plan to improve cpyext more and more. Ideally, we would like to reach a point in which cpyext-intensive programs run at worst at the same speed of CPython.

The other big topic of the sprint was Armin and Maciej doing a lot of work on the unicode-utf8 branch: the goal of the branch is to always use UTF-8 as the internal representation of unicode strings. The advantages are various:
  • decoding a UTF-8 stream is super fast, as you just need to check that the stream is valid;
  • encoding to UTF-8 is almost a no-op;
  • UTF-8 is always more compact representation than the currently used UCS-4. It's also almost always more compact than CPython 3.5 latin1/UCS2/UCS4 combo;
  • smaller representation means everything becomes quite a bit faster due to lower cache pressure.
Before you ask: yes, this branch contains special logic to ensure that random access of single unicode chars is still O(1), as it is on both CPython and the current PyPy.
We also plan to improve the speed of decoding even more by using modern processor features, like SSE and AVX. Preliminary results show that decoding can be done 100x faster than the current setup.

In summary, this was a long and profitable sprint, in which we achieved lots of interesting results. However, what we liked even more was the privilege of doing commits from awesome places such as the top of Table Mountain:

Our sprint venue today #pypy pic.twitter.com/o38IfTYmAV — Ronan Lamy (@ronanlamy) 4 ottobre 2017
The panorama we looked at instead of staring at cpyext code
Categories: FLOSS Project Planets

GCompris at KDE-edu sprint 2017

Planet KDE - Wed, 2017-10-18 09:18

Ten days ago, I spent a week-end in Berlin with a group of KDE friends to have a KDE-edu sprint. I didn’t blog about it yet because we planned to make a group post to summarize the event, but since it takes some time, I decided to write a quick personal report too.

The sprint was hosted in Endocode offices, which was a very nice place to work together.

Of course I came mostly because of GCompris, but the goal in the end was more to work together to try to redefine the goal and direction of KDE-edu and its website, and to work together on different tasks.

I added appstream links for all KDE-edu apps on their respective pages on KDE website. Those appstream links can be used to install directly applications from linux appstores supporting this standard.
On a side note, we thought it is a bit weird to be redirected from the KDE-edu website to KDE.org when looking at application info. This is one of the things that would need some refactoring. Actually, we discussed a lot about the evolution needed for the website. I guess all the details about this discussion will be on the group-post report, but to give you an idea, I would summarize it as : let’s make KDE-edu about how KDE-applications can be used in educational context, rather than just a collection of specific apps. A lot of great ideas to work on!

For GCompris, I was very happy to can meet Rishabh, who did some work on the server part. I could test the branch with him, and discussed about what needs to be done. Next, I fixed and improved the screenshots available for our appdata info, and started to look at building a new package on Mac with Sanjiban.

I also cleaned an svg file of Ktuberling to help Albert who worked on buiding it for Android.

In the end, I would say it was a productive week-end. Many thanks to KDE e.V. for the travel support, and to Endocode for hosting the event and providing cool drinks.

Categories: FLOSS Project Planets

Cobe.io: Pruning a Private Docker Registry

Planet Python - Wed, 2017-10-18 08:00
Why do we need a private Docker registry?

We currently use Jenkins to run our build and test process and we currently have two slaves running on our Jenkins cluster. When a pull request is created/updated, our process builds the Docker files that are stored within the code repository. Once a slave has built a Docker image it's ideal that the other slave can access the newly built image. One way to achieve this is to have a centrally accessible registry.

How to run a Docker registry

This is relatively easy step as there is a Docker registry image available on Docker Hub. Currently our registry is running on our Jenkins master server. Execute this command to run it:

docker run -d -p 5000:5000 --name registry \ --restart=unless-stopped -e REGISTRY_STORAGE_DELETE_ENABLED=true registry

I'll go through each command option briefly:

docker run
Uses Docker to run a container based on an image.
-d
Run in detatched mode.
-p 5000:5000
Expose port 5000 from the current host into the registry container.
--name registry
Names the container to make it easier to reference.
--restart=unless-stopped
Tells Docker to keep this container running unless manually stopped.
-e REGISTRY_STORAGE_DELETE_ENABLED=true
Configures the registry to allow DELETE requests.
registry
The image to run from Docker Hub.

This will run a Docker registry that allows delete requests on port 5000.

Persisting the registry

When the image is restarted it loses its images that it stores. This is solved by using a Docker volume to store the images.

docker volume create registry-vol

And adding the following argument to the Docker run command above:

-v registry-vol:/var/lib/registry:rw

So the full command is now:

docker run -d -p 5000:5000 --name registry \ --restart=unless-stopped \ -e REGISTRY_STORAGE_DELETE_ENABLED=true \ -v registry-vol:/var/lib/registry:rw registry Clearing out unused images

As all of the images that Jenkins pushes are tagged as latest our goal is to search through all of the repositories in the registry and delete all of the images tagged as latest.

To do this we first get all of the repositories using this method.

REGISTRY_URL = "https://registry:5000/v2/" def get_repositories(): resp = requests.get(REGISTRY_URL + "_catalog") return resp.json()['repositories']

For each of our repositories we get a list of tags.

def get_tags(repository): resp = requests.get(REGISTRY_URL + repository + "/tags/list") return resp.json()['tags'] if json_resp['tags'] else []

In order to delete an image we need its digest.

def get_digest(repository, tag): url = "{}{}/manifests/{}".format(JENKINS_REGISTRY_URL, repository, tag) headers = {"Accept": "application/vnd.docker.distribution.manifest.v2+json"} resp = requests.get(url, headers=headers) return resp.headers.get("Docker-Content-Digest")

And we use this method to submit a delete request.

def delete_digest(repository, digest): requests.delete(REGISTRY_URL + repo + "/manifests/" + digest)

So to tie all of this together we use this method.

def clear_registry_tag(tag="latest"): for repository in get_repositories(): for found_tag in get_tags(repository): if found_tag == tag: digest = get_digest(repository, found_tag) delete_digest(repository, digest)

For brevity error handling and printing have been removed but a full version of the Python script can be downloaded pruning-docker-registry.py.

Categories: FLOSS Project Planets

Amazee Labs: Take the Amazee Agile Agency Survey 2017

Planet Drupal - Wed, 2017-10-18 07:57
Take the Amazee Agile Agency Survey 2017

As part of my new role as Agile Consultant with Amazee Labs Zurich, I'm running a global survey to assess agile practices in our industry. Anyone working in an agency environment is welcome to fill out the survey!

Josef Dabernig Wed, 10/18/2017 - 13:57

Do you / does your agency work using defined agile methodologies such as Scrum and or Kanban? How do you fit theory into practice when it comes to working for different clients with different levels of understanding with regards to Agile practices at the same time?

Thank you for taking the survey before October 31 - I’m looking forward to report the findings in an upcoming blog post.

Categories: FLOSS Project Planets

Codementor: Introduction to Machine Learning with Python's Scikit-learn

Planet Python - Wed, 2017-10-18 07:33
This is a step-by-step walkthrough of a basic machine learning project, geared toward people with some knowledge of programming, but who don’t have much experience with machine learning.
Categories: FLOSS Project Planets

Ned Batchelder: How code slows as data grows

Planet Python - Wed, 2017-10-18 06:43

One of the parts of the vacation talk I did in September at Boston Python was about big-O notation. I've noticed that topic seems to be some kind of dividing line for people who feel insecure about not having a computer science degree. I wanted to explain it in simple practical terms so that people could understand it well enough to inform their choices during everyday coding.

I liked how it came out, so I wrote it up as a standalone piece: Big-O: how code slows as data grows.

Categories: FLOSS Project Planets

mailutils @ Savannah: Version 3.3 available

GNU Planet! - Wed, 2017-10-18 06:14

Mailutils version 3.3 is available for download. See the NEWS file, for information about changes in this version.

Categories: FLOSS Project Planets

Michal &#268;iha&#345;: Gammu 1.38.5

Planet Debian - Wed, 2017-10-18 06:00

Today, Gammu 1.38.5 has been released. After long period of bugfix only releases, this comes with several new noteworthy features.

The biggest feature probably is that SMSD can now handle USSD messages as well. Those are usually used for things like checking remaining credit, but it's certainly not limited to this. This feature has been contributed thanks to funding on BountySource.

You can read more information in the release announcement.

Filed under: Debian English Gammu

Categories: FLOSS Project Planets

Kay Hayen: Nuitka Release 0.5.28

Planet Python - Wed, 2017-10-18 05:00

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler. Please see the page "What is Nuitka?" for an overview.

This release has a focus on compatibility work and contains bug fixes and work to enhance the usability of Nuitka by integrating with distutils. The major improvement is that contractions no longer use pseudo functions to achieve their own local scope, but that there is now a dedicated structure for that representing an in-lined function.

Bug Fixes
  • Python3.6: Fix, async for was not yet implemented for async generators.
  • Fix, functions with keyword arguments where the value was determined to be a static raise could crash the compiler.
  • Detect using MinGW64 32 bits C compiler being used with 64 bits Python with better error message.
  • Fix, when extracting side effects of a static raise, extract them more recursively to catch expressions that themselves have no code generation being used. This fixes at least static raises in keyword arguments of a function call.
  • Compatibility: Added support for proper operation of `pkgutil.get_data by implementing get_data in our meta path based loader.
  • Compatibility: Added __spec__ module attribute was previously missing, present on Python3.4 and higher.
  • Compatibility: Made __loader__ module attribute set when the module is loading already.
  • Standalone: Resolve the @rpath and @loader_path from otool on MacOS manually to actual paths, which adds support for libraries compiled with that.
  • Fix, nested functions calling super could crash the compiler.
  • Fix, could not use --recurse-directory with arguments that had a trailing slash.
  • Fix, using --recurse-directory on packages that are not in the search crashed the compiler.
  • Compatibility: Python2 set and dict contractions were using extra frames like Python3 does, but those are not needed.
  • Standalone: Fix, the way PYTHONHOME was set on Windows had no effect, which allowed the compiled binary to access the original installation still.
  • Standalone: Added some newly discovered missing hidden dependencies of extension modules.
  • Compatiblity: The name mangling of private names (e.g. __var) in classes was applied to variable names, and function declarations, but not to classes yet.
  • Python3.6: Fix, added support for list contractions with await expressions in async generators.
  • Python3.6: Fix, async for was not working in async generators yet.
  • Fix, for module tracebacks, we output the module name <module name> instead of merely <module>, but if the module was in a package, that was not indicated. Now it is <module package.name>.
  • Windows: The cache directory could be unicode which then failed to pass as an argument to scons. We now encode such names as UTF-8 and decode in Scons afterwards, solving the problem in a generic way.
  • Standalone: Need to recursively resolve shared libraries with ldd, otherwise not all could be included.
  • Standalone: Make sure sys.path has no references to CPython compile time paths, or else things may work on the compiling machine, but not on another.
  • Standalone: Added various missing dependencies.
  • Standalone: Wasn't considering the DLLs directory for standard library extensions for freezing, which would leave out these.
  • Compatibility: For __future__ imports the __import__ function was called more than once.
Optimization
  • Contractions are now all properly inlined and allow for optimization as if they were fully local. This should give better code in some cases.
  • Classes are now all building their locals dictionary inline to the using scope, allowing for more compact code.
  • The dictionary API was not used in module template code, although it helps to generate more compact code.
New Features
  • Experimental support for building platform dependent wheel distribution.

    python setup.py --command-packages=nuitka.distutils clean -a bdist_nuitka

    Use with caution, this is incomplete work.

  • Experimental support for running tests against compiled installation with nose and py.test.

  • When specifiying what to recurse to, now patterns can be used, e.g. like this --recurse-not-to=*.tests which will skip all tests in submodules from compilation.

  • By setting NUITKA_PACKAGE_packagename=/some/path the __path__ of packages can be extended automatically in order to allow and load uncompiled sources from another location. This can be e.g. a tests sub-package or other plug-ins.

  • By default when creating a module, now also a module.pyi file is created that contains all imported modules. This should be deployed alongside the extension module, so that standalone mode creation can benefit from knowing the dependencies of compiled code.

  • Added option --plugin-list that was mentioned in the help output, but still missing so far.

  • The import tracing of the hints module has achieved experimental status and can be used to test compatibility with regards to import behavior.

Cleanups
  • Rename tree and codegen Helper modules to unique names, making them easier to work with.
  • Share the code that decides to not warn for standard library paths with more warnings.
  • Use the bool enum definition of Python2 which is more elegant than ours.
  • Move quality tools, autoformat, isort, etc. to the nuitka.tools.quality namespace.
  • Move output comparison tool to the nuitka.tools.testing namespace.
  • Made frame code generation capable of using nested frames, allowing the real inline of classes and contraction bodies, instead of "direct" calls to pseudo functions being used.
  • Proper base classes for functions that are entry points, and functions that are merely a local expression using return statements.
Tests
  • The search mode with pattern, was not working anymore.
  • Resume hash values now consider the Python version too.
  • Added test that covers using test runners like nose and py.test with Nuitka compiled extension modules.
Organizational
  • Added support for Scons 3.0 and running Scons with Python3.5 or higher. The option to specifiy the Python to use for scons has been renamed to reflect that it may also be a Python3 now. Only for Python3.2 to Python3.4 we now need another Python installation.
  • Made recursion the default for --recurse-directory with packages. Before you also had to tell it to recurse into that package or else it would only include the top level package, but nothing below.
  • Updated the man pages, correct mentions of its C++ to C and don't use now deprecated options.
  • Updated the help output which still said that standalone mode implies recursion into standard library, which is no longer true and even not recommended.
  • Added option to disable the output of .pyi file when creating an extension module.
  • Removed Ubuntu Wily package download, no longer supported by Ubuntu.
Summary

This release was done to get the fixes and new features out for testing. There is work started that should make generators use an explicit extra stack via pointer, and restore instruction state via goto dispatchers at function entry, but that is not complete.

This feature, dubbed "goto generators" will remove the need for fibers (which is itself a lot of code), reduce the memory footprint at run time for anything that uses a lot of generators, or coroutines.

Integrating with distutils is also a new thing, and once completed will make use of Nuitka for existing projects automatic and trivial to do. There is a lot missing for that goal, but we will get there.

Also, documenting how to run tests against compiled code, if that test code lives inside of that package, will make a huge difference, as that will make it easier for people to torture Nuitka with their own test cases.

And then of course, nested frames now mean that every function could be inlined, which was previously not possible due to collisions of frames. This will pave the route for better optimization in those cases in future releases.

The experimental features will require more work, but should make it easier to use Nuitka for existing projects. Future releases will make integrating Nuitka dead simple, or that is the hope.

And last but not least, now that Scons works with Python3, chances are that Nuitka will more often work out the of the box. The older Python3 versions that still retain the issue are not very widespread.

Categories: FLOSS Project Planets

Steinar H. Gunderson: Introducing Narabu, part 1: Introduction

Planet Debian - Wed, 2017-10-18 04:25

Narabu is a new intraframe video codec, from the Japanese verb narabu (並ぶ), which means to line up or be parallel.

Let me first state straight up that Narabu isn't where I hoped it would be at this stage; the encoder isn't fast enough, and I have to turn my attention to other projects for a while. Nevertheless, I think it is interesting as a research project in its own right, and I don't think it should stop me from trying to write up a small series. :-)

In the spirit of Leslie Lamport, I'll be starting off with describing what problem I was trying to solve, which will hopefully make the design decisions a lot clearer. Subsequent posts will dive into background information and then finally Narabu itself.

I want a codec to send signals between different instances of Nageru, my free software video mixer, and also longer-term between other software, such as recording or playout. The reason is pretty obvious for any sort of complex configuration; if you are doing e.g. both a stream mix and a bigscreen mix, they will naturally want to use many of the same sources, and sharing them over a single GigE connection might be easier than getting SDI repeaters/splitters, especially when you have a lot of them. (Also, in some cases, you might want to share synthetic signals, such as graphics, that never existed on SDI in the first place.)

This naturally leads to the following demands:

  • Intraframe-only; every frame must be compressed independently. (This isn't strictly needed for all use cases, but is much more flexible, and common in any kind of broadcast.)
  • Need to handle 4:2:2 color, since that's what most capture sources give out, and we want to transmit the raw signals as much as possible. Fairly flexible in input resolution (divisible by 16 is okay, limited to only a given set of resolutions is not).
  • 720p60 video in less than one CPU core (ideally much less); the CPU can already pretty be busy with other work, like x264 encoding of the finished stream, and sharing four more inputs at the same time is pretty common. What matters is mostly a single encode+decode cycle, so fast decode doesn't help if the encoder is too slow.
  • Target bitrates around 100-150 Mbit/sec, at similar quality to MJPEG (ie. 45 dB PSNR for most content). Multiple signals should fit into a normal GigE link at the same time, although getting it to work over 802.11 isn't a big priority.
  • Both encoder and decoder robust to corrupted or malicious data; a dropped frame is fine, a crash is not.
  • Does not depend on uncommon or expensive hardware, or GPUs from a specific manufacturer.
  • GPLv3-compatible implementation. I already link to GPLv3 software, so I don't have a choice here; I cannot link to something non-free (and no antics with dlopen(), please).

There's a bunch of intraframe formats around. The most obvious thing to do would be to use Intel Quick Sync to produce H.264 (intraframe H.264 blows basically everything else out of the sky in terms of PSNR, and QSV hardly uses any power at all), but sadly, that's limited to 4:2:0. I thought about encoding the three color planes as three different monochrome streams, but monochrome is not supported either.

Then there's a host of software solutions. x264 can do 4:2:2, but even on ultrafast, it gobbles up an entire core or more at 720p60 at the target bitrates (mostly in entropy coding). FFmpeg has implementations of all kinds of other codecs, like DNxHD, CineForm, MJPEG and so on, but they all use much more CPU for encoding than the target. NDI would seem to fit the bill exactly, but fails the licensing check, and also isn't robust to corrupted or malicious data. (That, and their claims about video quality are dramatically overblown for any kinds of real video data I've tried.)

So, sadly, this leaves only really one choice, namely rolling my own. I quickly figured I couldn't beat the world on CPU video codec speed, and didn't really want to spend my life optimizing AVX2 DCTs anyway, so again, the GPU will come to our rescue in the form of compute shaders. (There are some other GPU codecs out there, but all that I've found depend on CUDA, so they are NVIDIA-only, which I'm not prepared to commit to.) Of course, the GPU is quite busy in Nageru, but if one can make an efficient enough codec that one stream can work at only 5% or so of the GPU (meaning 1200 fps or so), it wouldn't really make a dent. (As a spoiler, the current Narabu encoder isn't there for 720p60 on my GTX 950, but the decoder is.)

In the next post, we'll look a bit at the GPU programming model, and what it means for how our codec needs to look like on the design level.

Categories: FLOSS Project Planets

Agiledrop.com Blog: AGILEDROP: Drupal events in 4th quarter of the year

Planet Drupal - Wed, 2017-10-18 04:18
We've stepped into the last quarter of the year, but in Drupal community is still much going on. We've made a list of DrupalCamps and summits that are still available to attend. Drupal events are bringing together Drupal developers, themers, end users and those interested in learning more about Drupal for talks, sessions and collaborative discussions.  Drupal Summit Tokyo  Fukurasia Shinagawa Crystal Square, Tokyo, Japan 19. October 2017 9:00-19:00 Largest Drupal event in Japan will host more than 15 strategies and technical sessions, starting with a session of a formal digital director of… READ MORE
Categories: FLOSS Project Planets

OSTraining: Importing data from a CSV file into Drupal 8 with the Content Import module

Planet Drupal - Wed, 2017-10-18 03:00

Sometimes you would like to import a huge volume of data from a CSV file into Drupal. Maybe from another CMS. Maybe from a spreadsheet. But there is no such functionality in Drupal 8 core. 

To import your data from a CSV file, you need to install and enable the contributed module "Content Import". In this tutorial, you are going to import five content items of the content type Customer.

Categories: FLOSS Project Planets

Mahmoud Hashemi: Maintainerati 2017: GitHub Design

Planet Python - Wed, 2017-10-18 01:00

Last week I attended a Maintainerati event, an unconference/mini-summit for maintainers of popular software, run as a prelude to the GitHub Universe conference. After being brought up to speed on this year's secret handshake of the software elite, I had a great time in the documentation breakout group, as well as moderating a lively discussion on diversity in open-source, both of which deserve their own write-ups at some point.

Once those were through and coffee breaks were had, what I consider the main event was upon us: An opportunity to discuss with GitHub designers and developers all the different ways projects use GitHub, and how GitHub might improve to match those use cases. I think these interactions have the most direct potential to bear fruit, so in my excitement I wrote a bunch of the proceedings down:

ContentsDigested emails

Discussion is one of the greatest things about GitHub, but as Jupyter developer Ian Rose brought up, an email for every comment can be overwhelming. Daily or weekly digests for issues, or even for all your GitHub activity would be a huge improvement, especially for more lightweight users of GitHub. This may not strike subscribers of The Weeklypedia as a surprise, but I am a big fan of email digests.

More control over engagement levels could open a great new avenue for driving traffic for GitHub, too.

Star spectrum

Right now you can either star repos or watch them, effectively getting no notifications or all of them.

I'd estimate about a third of the repos I star look interesting, but haven't yet reached the point where I'd use or contribute to them. So they mostly get starred and forgotten.

A friend of mine started a little project called Starminder, which emails a nightly selection of five of my starred repos. I've been having a grand time revisiting these old stars and seeing how far they've come, even reminding me of features I was waiting to build.

And while I love Nik's work, instead of relying on Starminder, it would be way better if I could tell GitHub roughly how often I'd like updates on a project, and then get Pulse-like info delivered to my inbox on a weekly or monthly basis.

Commit activity, high-traffic issues, and especially new tags/releases are all things I'd be very excited to get personalized, periodic updates on, without having to get every single notification as a separate email.

One off-the-cuff idea I had was to establish some sort of star gradient, with the basic star without notifications being an option on one end of the opt-in engagement spectrum, and full-blown, every-notification "Watching" on the other. Could there be one Star dropdown to rule them all?

Code review permissions

Requiring code review before merging is a pretty smart idea for any rigorous project, and now GitHub supports it natively. However, only developers with write permissions can actually perform a code review. Here are some real-life use cases that demonstrate why this is less than ideal:

  • A senior developer not involved with the project files an issue requesting a feature. I would like them to review the implementation to ensure it does what they want. The senior developer has a busy schedule and doesn't want to join the project and get a bunch of notifications, but would be qualified to review the code.
  • A novice developer finds an problem in the documentation, they could review the new documentation for clarity. Their lack of experience makes them best qualified to review.
  • The core maintainer implements a feature, but is actually the only developer on the project. Requesting a code review from non-project-member peers is a great way to get them to look at the code and become more involved with the project going forward.

For bonus points all permissions could have an option to be time-limited. Designated reviewr and expiration possibilities notwithstanding, I think the best flow would include the ability to add someone to a specific PR as reviewer, without giving them any project-wide permissions.

Dashboard improvements

I'm probably weird for doing this, but I habitually visit the normal github.com logged-in landing page, aka the dashboard, several times a day. Now, for the few of you who share my habit probably noticed, there's a new Discover tab, offering personalized suggestions of repos to star.

The event stream stayed mostly the same, however, as it has for many years. But despite its maturity there are a couple events that surprisingly don't show up anywhere, even when the dashboard seems like a natural fit:

  • Follows - I have the better part of a thousand followers, but I can't remember if I've ever seen a notification about this. They seem like nice folks!
  • Stars on org-owned repos - There are several repos I maintain and watch, but for which I've never seen on-dash notifications. What do they all have in common? They're all owned by organizations (e.g., python-hyper). Other types of notifications show up, but not stars.
  • Watches - Not sure I've ever gotten a notification for someone watching one of my repos, even though they're probably more interested in collaboration than the average stargazer.

Any or all of these would certainly make my github.com itch yield more interesting results, and I'm sure there are some enhancements I've missed, too!

Thanks!

Just wanted to say thanks to GitHub for putting together such a great event. Whether or not any of these features materializes in the near future, it was so nice to meet up with old friends and make some new ones, too.

Focused, cross-technology encounters like these are all too rare. For my Python readers, let this serve as a reminder to get out and interact with other stacks. Python's strength is its integrative nature, and I think that can be a strength for us Pythonists as well.

In any case, thanks for the event, GitHub! Hope to see you again next year!

Categories: FLOSS Project Planets

Acro Media: Video: Checkout in Drupal Commerce 2.x is Configurable for any Order Type

Planet Drupal - Tue, 2017-10-17 23:00

A checkout is a pretty fundamental part of a commerce system. So the fact that Commerce 2.x has a checkout is not really news. But it’s what you can do with the checkout that makes 2.x special.

You can now configure the checkout workflow. You can opt to ask for billing information, shipping information, certificates, registration details, etc. There’s lots of different data that can change depending on the type of product you sell. If you sell digital products, for instance, you don’t need shipping information. If you sell course registrations, you might require pre-existing certificates. Maybe you do both, so you need to configure multiple types of checkouts.

And that’s easy to do. For the most part, it’s a matter of dragging and dropping options. You can add or remove pieces pretty easily. If you need something really custom, like if you need to validate a safety certificate against a third party, you might need a developer to build that functionality. But otherwise it’s a fairly simple process.

You can also integrate into any part of the checkout. Maybe you do something when you add to cart, or when you complete the order. Maybe you even go off-site to pay through PayPal or register through Eventbrite and then come back. You can hook into any step you need in order to get those things done.

Categories: FLOSS Project Planets
Syndicate content