Feeds

Apache Spark (.) is increasingly becoming popular in the field of Data Sciences because of its ability to deal with the huge datasets and the capability to run computations in memory which is...

I am delighted to share with folks the results of a project I have been helping out on for the last few months: implementation of "BigInt" in Firefox, which is finally shipping in Firefox 68 (beta).

what's a bigint?

BigInts are a new kind of JavaScript primitive value, like numbers or strings. A BigInt is a true integer: it can take on the value of any finite integer (subject to some arbitrarily large implementation-defined limits, such as the amount of memory in your machine). This contrasts with JavaScript number values, which have the well-known property of only being able to precisely represent integers between -253 and 253.

BigInts are written like "normal" integers, but with an n suffix:

var a = 1n; var b = a + 42n; b << 64n // result: 793209995169510719488n

With the bigint proposal, the usual mathematical operations (+, -, *, /, %, <<, >>, **, and the comparison operators) are extended to operate on bigint values. As a new kind of primitive value, bigint values have their own typeof:

typeof 1n // result: 'bigint'

Besides allowing for more kinds of math to be easily and efficiently expressed, BigInt also allows for better interoperability with systems that use 64-bit numbers, such as "inodes" in file systems, WebAssembly i64 values, high-precision timers, and so on.

You can read more about the BigInt feature over on MDN, as usual. You might also like this short article on BigInt basics that V8 engineer Mathias Bynens wrote when Chrome shipped support for BigInt last year. There is an accompanying language implementation article as well, for those of y'all that enjoy the nitties and the gritties.

can i ship it?

To try out BigInt in Firefox, simply download a copy of Firefox Beta. This version of Firefox will be fully released to the public in a few weeks, on July 9th. If you're reading this in the future, I'm talking about Firefox 68.

BigInt is also shipping already in V8 and Chrome, and my colleague Caio Lima has an project in progress to implement it in JavaScriptCore / WebKit / Safari. Depending on your target audience, BigInt might be deployable already!

thanks

I must mention that my role in the BigInt work was relatively small; my Igalia colleague Robin Templeton did the bulk of the BigInt implementation work in Firefox, so large ups to them. Hearty thanks also to Mozilla's Jan de Mooij and Jeff Walden for their patient and detailed code reviews.

Thanks as well to the V8 engineers for their open source implementation of BigInt fundamental algorithms, as we used many of them in Firefox.

Finally, I need to make one big thank-you, and I hope that you will join me in expressing it. The road to ship anything in a web browser is long; besides the "simple matter of programming" that it is to implement a feature, you need a specification with buy-in from implementors and web standards people, you need a good working relationship with a browser vendor, you need willing technical reviewers, you need to follow up on the inevitable security bugs that any browser change causes, and all of this takes time. It's all predicated on having the backing of an organization that's foresighted enough to invest in this kind of long-term, high-reward platform engineering.

In that regard I think all people that work on the web platform should send a big shout-out to Tech at Bloomberg for making BigInt possible by underwriting all of Igalia's work in this area. Thank you, Bloomberg, and happy hacking!

Although the main conference starts on Wednesday, July 10th, there’s already so much to do for attendees with the main conference ticket on Monday 8th and Tuesday 9th.

Beginners’ Day and Sponsored Trainings

You can come to the workshops and trainings venue at FHNW Campus Muttenz and:

• pick up your conference badge
• attend the Beginners’ Day workshop
• attend the sponsored trainings

If you want to attend other workshops and trainings, you’ll need a separate training ticket or combined ticket.

Details on the Beginners’ Day workshop and the sponsored trainings will be announced separately.

Catering on training days not included

Since we have to budget carefully, the lunch and coffee breaks are not included, if you don’t have a training ticket or combined ticket.

To not keep you hungry, we have arranged that you can buy lunch coupons (price to be announced later). You can also go to the grocery store at the ground floor. For coffee breaks you can go to the ground floor, to the 12th floor of the FHNW building, or outside at the beach bar (nice weather only) and buy drinks.

Enjoy,
–
EuroPython 2019 Team
https://ep2019.europython.eu/
https://www.europython-society.org/

Software testing, if done right, is done all the time, throughout the whole life of a software project. This is different than the verification and validation of a classical model of QA teams. It's more of a collaborative model that actually tries to help get great software out the door faster and iterate quicker.

One of the people at the forefront of this push is Alan Page. Alan and his podcast cohost Brent Jensen tried to boil down what modern testing looks like in the Modern Testing Principles.

I've got Alan here today, to talk about the principles, and also to talk about this transition from classical QA to testing specialists being embedded in software teams and then to software teams doing their own testing.

But that only barely scratches the surface of what we cover. I think you'll learn a lot from this discussion.

The seven principles of Modern Testing:

1. Our priority is improving the business.
2. We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system.
3. We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures.
4. We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.
5. We believe that the customer is the only one capable to judge and evaluate the quality of our product
6. We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact.
7. We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist.

Special Guest: Alan Page.

Sponsored By:

• Patreon Supporters: Help support the show with as little as $1 per month. Funds help pay for expenses associated with the show.

Support Test & Code - Software Testing, Development, Python

Links:

• Tooth of the Weasel – notes and rants about software and software quality
• AB Testing – Alan and Brent talk about Modern Testing – including Agile, Data, Leadership, and more.
• Modern Testing Principles
• The Lean Startup

<p>Software testing, if done right, is done all the time, throughout the whole life of a software project. This is different than the verification and validation of a classical model of QA teams. It&#39;s more of a collaborative model that actually tries to help get great software out the door faster and iterate quicker. </p> <p>One of the people at the forefront of this push is Alan Page. Alan and his podcast cohost Brent Jensen tried to boil down what modern testing looks like in the Modern Testing Principles.</p> <p>I&#39;ve got Alan here today, to talk about the principles, and also to talk about this transition from classical QA to testing specialists being embedded in software teams and then to software teams doing their own testing.</p> <p>But that only barely scratches the surface of what we cover. I think you&#39;ll learn a lot from this discussion.</p> <p><strong>The seven principles of <a href="http://moderntesting.org" rel="nofollow">Modern Testing</a>:</strong> </p> <ol> <li>Our priority is improving the business.</li> <li>We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system.</li> <li>We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures.</li> <li>We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.</li> <li>We believe that the customer is the only one capable to judge and evaluate the quality of our product</li> <li>We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact.</li> <li>We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist. </li> </ol><p>Special Guest: Alan Page.</p><p>Sponsored By:</p><ul><li><a href="https://www.patreon.com/testpodcast" rel="nofollow">Patreon Supporters</a>: <a href="https://www.patreon.com/testpodcast" rel="nofollow">Help support the show with as little as $1 per month. Funds help pay for expenses associated with the show.</a></li></ul><p><a href="https://www.patreon.com/testpodcast" rel="payment">Support Test & Code - Software Testing, Development, Python</a></p><p>Links:</p><ul><li><a href="https://angryweasel.com/blog/" title="Tooth of the Weasel – notes and rants about software and software quality" rel="nofollow">Tooth of the Weasel – notes and rants about software and software quality</a></li><li><a href="https://www.angryweasel.com/ABTesting/" title="AB Testing – Alan and Brent talk about Modern Testing – including Agile, Data, Leadership, and more." rel="nofollow">AB Testing – Alan and Brent talk about Modern Testing – including Agile, Data, Leadership, and more.</a></li><li><a href="https://www.angryweasel.com/ABTesting/modern-testing-principles/" title="Modern Testing Principles" rel="nofollow">Modern Testing Principles</a></li><li><a href="https://amzn.to/2WigLM2" title="The Lean Startup" rel="nofollow">The Lean Startup</a></li></ul>

This tutorial follows a similar tutorial from the web. I tested that tutorial to see if it works. This tutorial is focused on GitHub but can also be used independently on another python platform. You need a GitHub Pro, your personal account: With GitHub Pro, your personal account gets unlimited public and private repositories with unlimited collaborators. In addition to the features available

I recently setup a desktop computer with two SSDs using a software RAID1 and full-disk encryption (i.e. LUKS). Since this is not a supported configuration in Ubuntu desktop, I had to use the server installation medium.

This is my version of these excellent instructions.

Server installer

Start by downloading the alternate server installer and verifying its signature:

1. Download the required files:

   wget http://cdimage.ubuntu.com/ubuntu/releases/bionic/release/ubuntu-18.04.2-server-amd64.iso wget http://cdimage.ubuntu.com/ubuntu/releases/bionic/release/SHA256SUMS wget http://cdimage.ubuntu.com/ubuntu/releases/bionic/release/SHA256SUMS.gpg

2. Verify the signature on the hash file:

   $ gpg --keyid-format long --keyserver hkps://keyserver.ubuntu.com --recv-keys 0xD94AA3F0EFE21092 $ gpg --verify SHA256SUMS.gpg SHA256SUMS gpg: Signature made Fri Feb 15 08:32:38 2019 PST gpg: using RSA key D94AA3F0EFE21092 gpg: Good signature from "Ubuntu CD Image Automatic Signing Key (2012) <cdimage@ubuntu.com>" [undefined] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 8439 38DF 228D 22F7 B374 2BC0 D94A A3F0 EFE2 1092

3. Verify the hash of the ISO file:

   $ sha256sum ubuntu-18.04.2-server-amd64.iso a2cb36dc010d98ad9253ea5ad5a07fd6b409e3412c48f1860536970b073c98f5 ubuntu-18.04.2-server-amd64.iso $ grep ubuntu-18.04.2-server-amd64.iso SHA256SUMS a2cb36dc010d98ad9253ea5ad5a07fd6b409e3412c48f1860536970b073c98f5 *ubuntu-18.04.2-server-amd64.iso

Then copy it to a USB drive:

dd if=ubuntu-18.04.2-server-amd64.iso of=/dev/sdX

and boot with it.

Inside the installer, use manual partitioning to:

1. Configure the physical partitions.
2. Configure the RAID array second.
3. Configure the encrypted partitions last

Here's the exact configuration I used:

• /dev/sda1 is 512 MB and used as the EFI parition
• /dev/sdb1 is 512 MB but not used for anything
• /dev/sda2 and /dev/sdb2 are both 4 GB (RAID)
• /dev/sda3 and /dev/sdb3 are both 512 MB (RAID)
• /dev/sda4 and /dev/sdb4 use up the rest of the disk (RAID)

I only set /dev/sda2 as the EFI partition because I found that adding a second EFI partition would break the installer.

I created the following RAID1 arrays:

• /dev/sda2 and /dev/sdb2 for /dev/md2
• /dev/sda3 and /dev/sdb3 for /dev/md0
• /dev/sda4 and /dev/sdb4 for /dev/md1

I used /dev/md0 as my unencrypted /boot partition.

Then I created the following LUKS partitions:

• md1_crypt as the / partition using /dev/md1
• md2_crypt as the swap partition (4 GB) with a random encryption key using /dev/md2

Post-installation configuration

Once your new system is up, sync the EFI partitions using DD:

dd if=/dev/sda1 of=/dev/sdb1

and create a second EFI boot entry:

efibootmgr -c -d /dev/sdb -p 1 -L "ubuntu2" -l \EFI\ubuntu\shimx64.efi

Ensure that the RAID drives are fully sync'ed by keeping an eye on /prod/mdstat and then reboot, selecting "ubuntu2" in the UEFI/BIOS menu.

Once you have rebooted, remove the following package to speed up future boots:

apt purge btrfs-progs

To switch to the desktop variant of Ubuntu, install these meta-packages:

apt install ubuntu-desktop gnome

then use debfoster to remove unnecessary packages (in particular the ones that only come with the default Ubuntu server installation).

Fixing booting with degraded RAID arrays

Since I have run into RAID startup problems in the past, I expected having to fix up a few things to make degraded RAID arrays boot correctly.

I did not use LVM since I didn't really feel the need to add yet another layer of abstraction of top of my setup, but I found that the lvm2 package must still be installed:

apt install lvm2

with use_lvmetad = 0 in /etc/lvm/lvm.conf.

Then in order to automatically bring up the RAID arrays with 1 out of 2 drives, I added the following script in /etc/initramfs-tools/scripts/local-top/cryptraid:

#!/bin/sh PREREQ="mdadm" prereqs() { echo "$PREREQ" } case $1 in prereqs) prereqs exit 0 ;; esac mdadm --run /dev/md0 mdadm --run /dev/md1 mdadm --run /dev/md2

before making that script executable:

chmod +x /etc/initramfs-tools/scripts/local-top/cryptraid

and refreshing the initramfs:

update-initramfs -u -k all Disable suspend-to-disk

Since I use a random encryption key for the swap partition (to avoid having a second password prompt at boot time), it means that suspend-to-disk is not going to work and so I disabled it by putting the following in /etc/initramfs-tools/conf.d/resume:

RESUME=none

and by adding noresume to the GRUB_CMDLINE_LINUX variable in /etc/default/grub before applying these changes:

update-grub update-initramfs -u -k all Test your configuration

With all of this in place, you should be able to do a final test of your setup:

1. Shutdown the computer and unplug the second drive.
2. Boot with only the first drive.
3. Shutdown the computer and plug the second drive back in.

4. Boot with both drives and re-add the second drive to the RAID array:

   mdadm /dev/md0 -a /dev/sdb3 mdadm /dev/md1 -a /dev/sdb4 mdadm /dev/md2 -a /dev/sdb2

5. Wait until the RAID is done re-syncing and shutdown the computer.

6. Repeat steps 2-5 with the first drive unplugged instead of the second.
7. Reboot with both drives plugged in.

At this point, you have a working setup that will gracefully degrade to a one-drive RAID array should one of your drives fail.

When you need to create survey style forms in Drupal 8 Webform is the clear winner. It's powerful enough to create all sorts of forms and you can even give it to your editor so they can create their own, after a little training, of course.

One part of Webform which I like is the ability to define conditional logic. For example, you can show or hide a text field based off a value from another element. You can also make an element conditionally required. It's a very useful part of Webform, and you do all of this through a UI, no custom code.

Defining simple conditional logic, check if element value has a single value, is pretty straightforward. But when you have to deal with multiple values, this is where things get tricky.

In this issue of Wing Tips we take a quick look at Wing Pro's remote development capabilities.

Setting up SSH Access

Wing Pro's remote development support requires using an SSH public/private key pair and SSH agent rather than entering a password each time you connect. This is more secure and convenient, and it allows Wing to seamlessly re-establish the remote connection as needed over time. If you currently enter a password (other than a 2-factor authentication card selector) each time you ssh to the remote host, then please see SSH Setup Details for instructions.

Creating a Remote Project

To create a Wing Pro project that works with a remote host, select New Project from the Project menu and use Connect to Remote Host via SSH as the project type. You will need to choose an Identifier` that Wing uses to refer to the host, and enter the Host Name either as an IP address or name (optionally in username@hostname form).

If python is not on the PATH on your remote host, or is not the Python you want to use, then you will also need to paste the full path to Python into the Python Executable field. This is typically the value printed by import sys; print(sys.executable) in the Python that you want to use.

You will only rarely need to specify any of the other values in a remote host configuration. For now, leave them set to their default values. For example:

Once you submit this dialog, Wing will probe the remote host and install the remote agent. When this is complete, you should see a dialog with some information about the remote host:

You can now point Wing to your remotely stored source code with Add Existing Directory in the Project menu. When this is complete, save the project to local disk with Save Project in the Project menu.

That's all there is to it!

At this point all of Wing's features should work with files on the remote host, including editing, debugging, unit testing, working in the Python Shell (after restarting from its Options menu), using version control, searching, code navigation, running processes from the OS Commands tool, and so forth.

How it Works

Using New Project takes care of a few steps that can also be done manually, in the event that a project or remote host connection needs to be reconfigured:

(1) Your remote host configuration can be viewed and edited from Remote Hosts in the Project menu. From here, remote host configurations can also be marked as shared, so they can be reused by multiple projects or used to open projects that are stored on the remote host.

(2) Pointing a project at a remote host is done by changing the Python Executable under the Environment tab in Project Properties to Remote and selecting the remote host configuration. The remote host configuration determines which Python is used on the remote host.

Further Reading

If your code runs in a web server or other framework on the remote host, you will need to initiate debug using Wing's wingdbstub.py module as described in the section Initiating Debug in Remote Web Development, or check out the How-Tos for details on using Wing Pro with specific frameworks and tools.

For detailed documentation on Wing Pro's remote development capabilities see Remote Development in the product manual.

Getting Help

As always, don't hesitate to email support@wingware.com or post on the Q&A Forum for help!

That's it for now! We'll be back next week with more Wing Tips for Wing Python IDE.

Welcome

OSI Affiliate Member Powering Potential Inc. (PPI) is currently preparing for their annual fundraising event scheduled for Wednesday, June 5, 2019, from 6 to 8:30 p.m. at NoMad Studio, located at 29 W. 39th Street, 10th Floor, in New York City.

This year PPI celebrates their 10 Year Partnership with the Segal Family Foundation. The close, long-time relationship has been a key factor in the amazing progress PPI has made in bringing their “Educating through Technology” programs to the rural students in Tanzania.

Proceeds from this year’s event will go towards the Sazira Secondary School SPARC+ Lab Upgrade impacting 800+ students in rural Tanzania: an ambitious project needing $23,500. While this is significant, The Collegiate Churches of New York recently awarded Powering Potential a generous grant of $13,000 towards this goal.

PPI has an incredible event planned for their guests. Back by popular demand, Tanzanian dancers performing traditional dance led by Justa Lujwangana, CEO and founder of Curious on Tanzania will provide entertainment for the evening. A buffet will also feature authentic Tanzanian dishes based on menus from Taste of Tanzania by Miriam Malaquais. The author has donated twenty of her books for sale at the event with proceeds going to PPI.

Along with wine and soft drinks, a classic Tanzanian cocktail will be available to guests for a donation. The Dawa is a mixture of vodka, lime juice and honey. The word “dawa” is Swahili for medicine.

This year's event will also feature a Silent Auction with a variety of exciting items and services up for bidding.

Admission is $70 for early bird admission. Regular admission is $90 at the door.

Visit Poweringpotential.org to donate and reserve your space at this event.

Asante sana (thank you)!

As the maintainer of the mime-support in Debian, I would like to give Kudos to Petter Reinholdtsen, who just opened a ticket at the IANA to create a text/vnd.sosi media type. May his example be followed by others!

• allism: an introduction to a little-known condition archive.org
• The Genderbread Person | A free online resource for understanding gender identity, gender expression, and anatomical sex. archive.org
• Gender Construction Kit: The UK guide to changing things linked to gender archive.org
• The Badger's Sett: The Care and Feeding of Your Extrovert archive.org

• allism: an introduction to a little-known condition archive.org
• The Genderbread Person | A free online resource for understanding gender identity, gender expression, and anatomical sex. archive.org
• Gender Construction Kit: The UK guide to changing things linked to gender archive.org
• The Badger's Sett: The Care and Feeding of Your Extrovert archive.org

GNU Parallel 20190522 ('Akihito') has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/

GNU Parallel is 10 years old in a year on 2020-04-22. You are here by invited to a reception on Friday 2020-04-17.

See https://www.gnu.org/software/parallel/10-years-anniversary.html

Quote of the month:

Amazingly useful script!
-- unxusr@reddit.com

New in this release:

• --group-by groups lines depending on value of a column. The value can be computed.

• How to compress (bzip/gzip) a very large text quickly? https://medium.com/@gchandra/how-to-compress-bzip-gzip-a-very-large-text-quickly-27c11f4c6681

• Simple tutorial to install & use GNU Parallel https://medium.com/@gchandra/simple-tutorial-to-install-use-gnu-parallel-79251120d618

• Introducing Parallel into Shell https://petelawson.com/post/parallel-in-shell/

• Bug fixes and man page updates.

Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html

GNU Parallel - For people who live life in the parallel lane.

About GNU Parallel

GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.

If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.

GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:
(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash

Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.

When using programs that use GNU Parallel to process data for publication please cite:

O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.

If you like GNU Parallel:

• Give a demo at your local user group/team/colleagues
• Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
• Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
• Request or write a review for your favourite blog or magazine
• Request or build a package for your favourite distribution (if it is not already there)
• Invite me for your next conference

If you use programs that use GNU Parallel for research:

• Please cite GNU Parallel in you publications (use --citation)

If GNU Parallel saves you money:

• (Have your company) donate to FSF https://my.fsf.org/donate/

About GNU SQL

GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.

The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.

When using GNU SQL for a publication please cite:

O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.

About GNU Niceload

GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.

Elisa is a music player developed by the KDE community that strives to be simple and nice to use. We also recognize that we need a flexible product to account for the different workflows and use-cases of our users.

We focus on a very good integration with the Plasma desktop of the KDE community without compromising the support for other platforms (other Linux desktop environments, Windows and Android).

We are creating a reliable product that is a joy to use and respects our users privacy. As such, we will prefer to support online services where users are in control of their data.

I am happy to announce the release of 0.4.0 version of the Elisa music player.

The new features are explained in the following posts New features in Elisa, New Features in Elisa: part 2 and Elisa 0.4 Beta Release and More New Features.

There have been a couple more changes not yet covered.

Improved Grid Views Elements

Nate Graham has reworked the grid elements (especially visible with the albums view).

I must confess that I was a bit uneasy with this change (it was a part mostly unchanged since the early versions). I am now very happy about this change.

Before After Getting Involved

I would like to thank everyone who contributed to the development of Elisa, including code contributions, testing, and bug reporting and triaging. Without all of you, I would have stopped working on this project.

New features and fixes are already being worked on. If you enjoy using Elisa, please consider becoming a contributor yourself. We are happy to get any kind of contributions!

We have some tasks that would be perfect junior jobs. They are a perfect way to start contributing to Elisa. There are more not yet reported here but reported in bugs.kde.org.

The flathub Elisa package allows an easy way to test this new release.

Elisa source code tarball is available here. There is no Windows setup. There is currently a blocking problem with it (no icons) that is being investigated. I hope to be able to provide installers for later bugfix versions.

The phone/tablet port project could easily use some help to build an optimized interface on top of Kirigami. It remains to be seen how to handle this related to the current desktop UI.

 

Running Python in the web browser has been getting a lot of attention lately. Shaun Taylor-Morgan knows what he’s talking about here – he works for Anvil, a full-featured application platform for writing full-stack web apps with nothing but Python. So I invited him to give us an overview and comparison of the open-source solutions for running Python code in your web browser.

In the past, if you wanted to build a web UI, your only choice was JavaScript.

That’s no longer true. There are quite a few ways to run Python in your web browser. This is a survey of what’s available.

SIX SYSTEMS

I’m looking at six systems that all take a different approach to the problem. Here’s a diagram that sums up their differences.

The x-axis answers the question: when does Python get compiled? At one extreme, you run a command-line script to compile Python yourself. At the other extreme, the compilation gets done in the user’s browser as they write Python code.

The y-axis answers the question: what does Python get compiled to? Three systems make a direct conversion between the Python you write and some equivalent JavaScript. The other three actually run a live Python interpreter in your browser, each in a slightly different way.

1. TRANSCRYPT

Transcrypt gives you a command-line tool you can run to compile a Python script into a JavaScript file.

You interact with the page structure (the DOM) using a toolbox of specialized Python objects and functions. For example, if you import document, you can find any object on the page by using document like a dictionary. To get the element whose ID is name-box, you would use document["name-box"]. Any readers familiar with JQuery will be feeling very at home.

Here’s a basic example. I wrote a Hello, World page with just an input box and a button:

<input id="name-box" placeholder="Enter your name"> <button id="greet-button">Say Hello</button>

To make it do something, I wrote some Python. When you click the button, an event handler fires that displays an alert with a greeting:

def greet(): alert("Hello " + document.getElementById("name-box").value + "!") document.getElementById("greet-button").addEventListener('click', greet)

I wrote this in a file called hello.py and compiled it using transcrypt hello.py. The compiler spat out a JavaScript version of my file, called hello.js.

Transcrypt makes the conversion to JavaScript at the earliest possible time – before the browser is even running. Next we’ll look at Brython, which makes the conversion on page load.

2. BRYTHON

Brython lets you write Python in script tags in exactly the same way you write JavaScript. Just as with Transcrypt, it has a document object for interacting with the DOM.

The same widget I wrote above can be written in a script tag like this:

<script type="text/python"> from browser import document, alert def greet(event): alert("Hello " + document["name-box"].value + "!") document["greet-button"].bind("click", greet) </script>

Pretty cool, huh? A script tag whose type is text/python!

There’s a good explanation of how it works on the Brython GitHub page. In short, you run a function when your page loads:

<body onload="brython()">

that transpiles anything it finds in a Python script tag:

<script type="text/python"></script>

which results in some machine-generated JavaScript that it runs using JS’s eval() function.

3. SKULPT

Skulpt sits at the far end of our diagram – it compiles Python to JavaScript at runtime. This means the Python doesn’t have to be written until after the page has loaded.

The Skulpt website has a Python REPL that runs in your browser. It’s not making requests back to a Python interpreter on a server somewhere, it’s actually running on your machine.

Skulpt does not have a built-in way to interact with the DOM. This can be an advantage, because you can build your own DOM manipulation system depending on what you’re trying to achieve. More on this later.

Skulpt was originally created to produce educational tools that need a live Python session on a web page (example: Trinket.io). While Transcrypt and Brython are designed as direct replacements for JavaScript, Skulpt is more suited to building Python programming environments on the web (such as the full-stack app platform, Anvil).

We’ve reached the end of the x-axis in our diagram. Next we head in the vertical direction: our final three technologies don’t compile Python to JavaScript, they actually implement a Python runtime in the web browser.

4. PYPY.JS

PyPy.js is a JavaScript implementation of a Python interpreter. The developers took a C-to-JavaScript compiler called emscripten and ran it on the source code of PyPy. The result is PyPy, but running in your browser.

Advantages: It’s a very faithful implementation of Python, and code gets executed quickly.

Disadvantages: A web page that embeds PyPy.js contains an entire Python interpreter, so it’s pretty big as web pages go (think megabytes).

You import the interpreter using <script> tags, and you get an object called pypyjs in the global JS scope.

There are three main functions for interacting with the interpreter. To execute some Python, run pypyjs.exec(<python code>). To pass values between JavaScript and Python, use pypyjs.set(variable, value) and pypyjs.get(variable).

Here’s a script that uses PyPy.js to calculate the first ten square numbers:

<script type="text/javascript"> pypyjs.exec( // Run some Python 'y = [x**2. for x in range(10)]' ).then(function() { // Transfer the value of y from Python to JavaScript pypyjs.get('y') }).then(function(result) { // Display an alert box with the value of y in it alert(result) }); </script>

PyPy.js has a few features that make it feel like a native Python environment – there’s even an in-memory filesystem so you can read and write files. There’s also a document object that gives you access to the DOM from Python.

The project has a great readme if you’re interested in learning more.

5. BATAVIA

Batavia is a bit like PyPy.js, but it runs bytecode rather than Python. Here’s a Hello, World script written in Batavia:

<script id="batavia-helloworld" type="application/python-bytecode"> 7gwNCkIUE1cWAAAA4wAAAAAAAAAAAAAAAAIAAABAAAAAcw4AAABlAABkAACDAQABZAEAUykCegtI ZWxsbyBXb3JsZE4pAdoFcHJpbnSpAHICAAAAcgIAAAD6PC92YXIvZm9sZGVycy85cC9uenY0MGxf OTc0ZGRocDFoZnJjY2JwdzgwMDAwZ24vVC90bXB4amMzZXJyddoIPG1vZHVsZT4BAAAAcwAAAAA= </script>

Bytecode is the ‘assembly language’ of the Python virtual machine – if you’ve ever looked at the .pyc files Python generates, that’s what they contain (Yasoob dug into some bytecode in a recent post on this blog). This example doesn’t look like assembly language because it’s base64-encoded.

Batavia is potentially faster than PyPy.js, since it doesn’t have to compile your Python to bytecode. It also makes the download smaller – around 400kB. The disadvantage is that your code needs to be written and compiled in a native (non-browser) environment, as was the case with Transcrypt.

Again, Batavia lets you manipulate the DOM using a Python module it provides (in this case it’s called dom).

The Batavia project is quite promising because it fills an otherwise unfilled niche – ahead-of-time compiled Python in the browser that runs in a full Python VM. Unfortunately, the GitHub repo’s commit rate seems to have slowed in the past year or so. If you’re interested in helping out, here’s their developer guide.

6. PYODIDE

Mozilla’s Pyodide was announced in April 2019. It solves a difficult problem: interactive data visualisation in Python, in the browser.

Python has become a favourite language for data science thanks to libraries such as NumPy, SciPy, Matplotlib and Pandas. We already have Jupyter Notebooks, which are a great way to present a data pipeline online, but they must be hosted on a server somewhere.

If you can put the data processing on the user’s machine, they avoid the round-trip to your server so real-time visualisation is more powerful. And you can scale to so many more users if their own machines are providing the compute.

It’s easier said than done. Fortunately, the Mozilla team came across a version of the reference Python implementation (CPython) that was compiled into WebAssembly. WebAssembly is a low-level compliment to JavaScript that performs closer to native speeds, which opens the browser up for performance-critical applications like this.

Mozilla took charge of the WebAssembly CPython project and recompiled NumPy, SciPy, Matplotlib and Pandas into WebAssembly too. The result is a lot like Jupyter Notebooks in the browser – here’s an introductory notebook.

It’s an even bigger download than PyPy.js (that example is around 50MB), but as Mozilla point out, a good browser will cache that for you. And for a data processing notebook, waiting a few seconds for the page to load is not a problem.

You can write HTML, MarkDown and JavaScript in Pyodide Notebooks too. And yes, there’s a document object to access the DOM. It’s a really promising project!

MAKING A CHOICE

I’ve given you six different ways to write Python in the browser, and you might be able to find more. Which one to choose? This summary table may help you decide.

There’s a more general point here too: the fact that there is a choice.

As a web developer, it often feels like you have to write JavaScript, you have to build an HTTP API, you have to write SQL and HTML and CSS. The six systems we’ve looked at make JavaScript seem more like a language that gets compiled to, and you choose what to compile to it (And WebAssembly is actually designed to be used this way).

Why not treat the whole web stack this way? The future of web development is to move beyond the technologies that we’ve always ‘had’ to use. The future is to build abstractions on top of those technologies, to reduce the unnecessary complexity and optimise developer efficiency. That’s why Python itself is so popular – it’s a language that puts developer efficiency first.

ONE UNIFIED SYSTEM

There should be one way to represent data, from the database all the way to the UI. Since we’re Pythonistas, we’d like everything to be a Python object, not an SQL SELECT statement followed by a Python object followed by JSON followed by a JavaScript object followed by a DOM element.

That’s what Anvil does – it’s a full-stack Python environment that abstracts away the complexity of the web. Here’s a 7-minute video that covers how it works.

Remember I said that it can be an advantage that Skulpt doesn’t have a built-in way to interact with the DOM? This is why. If you want to go beyond ‘Python in the browser’ and build a fully-integrated Python environment, your abstraction of the User Interface needs to fit in with your overall abstraction of the web system.

So Python in the browser is just the start of something bigger. I like to live dangerously, so I’m going to make a prediction. In 5 years’ time, more than 50% of web apps will be built with tools that sit one abstraction level higher than JavaScript frameworks such as React and Angular. It has already happened for static sites: most people who want a static site will use WordPress or Wix rather than firing up a text editor and writing HTML. As systems mature, they become unified and the amount of incidental complexity gradually minimises.

If you’re reading this in 2024, why not get in touch and tell me whether I was right?

Sometimes the list of release-critical bugs is overwhelming, and it’s hard to find something to tackle.

Today’s invitation is to review and perhaps upload the patch included in bug #928883.

In this article, we'll examine four ways to use Python to check whether a string contains a substring. Each has their own use-cases and pros/cons, some of which we'll briefly cover here:

1) The in Operator

The easiest way to check if a Python string contains a substring is to use the in operator. The in operator is used to check data structures for membership in Python. It returns a Boolean (either True or False) and can be used as follows:

fullstring = "StackAbuse" substring = "tack" if substring in fullstring: print "Found!" else: print "Not found!"

This operator is shorthand for calling an object's __contains__ method, and also works well for checking if an item exists in a list.

2) The String.index() Method

The String type in Python has a method called index that can be used to find the starting index of the first occurrence of a substring in a string. If the substring is not found, a ValueError exception is thrown, which can to be handled with a try-except-else block:

fullstring = "StackAbuse" substring = "tack" try: fullstring.index(substring) except ValueError: print "Not found!" else: print "Found!"

This method is useful if you need to know the position of the substring, as opposed to just its existence within the full string.

3) The String.find() Method

The String type has another method called find which is more convenient to use than index, because we don't need to worry about handling any exceptions. If find doesn't find a match, it returns -1, otherwise it returns the left-most index of the substring in the larger string.

fullstring = "StackAbuse" substring = "tack" if fullstring.find(substring) != -1: print "Found!" else: print "Not found!"

If you'd prefer to avoid the need to catch errors, then this method should be favored over index.

4) Regular Expressions (REGEX)

Regular expressions provide a more flexible (albeit more complex) way to check strings for pattern matching. Python is shipped with a built-in module for regular expressions, called re. The re module contains a function called search, which we can use to match a substring pattern as follows:

from re import search fullstring = "StackAbuse" substring = "tack" if search(substring, fullstring): print "Found!" else: print "Not found!"

This method is best if you are needing a more complex matching function, like case insensitive matching. Otherwise the complication and slower speed of regex should be avoided for simple substring matching use-cases.

About the Author

This article was written by Jacob Stopak, a software consultant and developer with passion for helping others improve their lives through code. Jacob is the creator of Initial Commit - a site dedicated to helping curious developers learn how their favorite programs are coded. Its featured project helps people learn Git at the code level.

PALO ALTO, Calif. - May 14, 2019 ​ - The Open Source Initiative (OSI), the founding organization of the open source software movement, is excited to announce the Affiliate Membership of the Open Source Hong Kong (OSHK). For ten years OSHK has worked across Asia to support open source communities, foster open source development, and increase the use of open source software, their recent OSI Membership highlights both organizations' desires to collaborate across communities.

“OSHK mission is promoting Open Source Software projects in Hong Kong and foster its development by connecting to the global open source community. In joining OSI as an Affiliate Member, OSHK connects with OSI, and other open source organizations, to support the promotion of open source,' said Sammy Fung, President of OSHK. "Open Source Software is not just about viewing the source code, it also guarantees the right to use the software, and modify it for our own use. By working together, I believe both organizations will be able to extend our reach and missions."

“We are excited to welcome OSHK as an OSI Affiliate Member,” said Molly de Blanc, OSI President. “The open source community truly is global, and their dedication to that idea is what inspires us as an organization. Our work for the future of open source is driven by that global community, and having the voices of OSHK in our affiliate membership helps us meet our goal in promoting and protecting open source and communities. We look forward to supporting their efforts and collaborating to help spread the message of open source even further.”

Currently, OSHK organizes the Hong Kong Open Source Conference (since 2013) and PyCon Hong Kong (beginning in 2015), with hopes to extend those event―and even add additional opportunities for open source communities to meet and collaborate. As a global organization the OSI is keen to help promote OSHK’s current events to both regional communities and our partners worldwide, as well as help develop new venues to raise awareness and adoption of open source software. As a growing segment in a large market, Asia promises to be a vital community in open source development and innovation. A partnership between OSI and OSHK will be critical in continued success.

The OSI Affiliate Member program allows non-profit and not-for-profit organizations to become OSI members. Full details of OSI Affiliate Membership and the names of the Affiliate Members are available on OSI's Affiliate Membership pages (www.opensource.org/affiliates) together with details for other non-profit and not-for profit organizations on how to join. Individuals can also become Members at www.opensource.org/join

About Open Source Hong Kong
Established in 2009, Open Source Hong Kong (OSHK) serves to foster and promote open source communities of and for developers, contributors, engineers, promoters, and users. OSHK develops and supports open source projects, and organizes conferences and events,within Hong Kong and across Asia, to encourage collaboration in the global open source community. To learn more about OSHK, visit: ​ https://opensource.hk/about/

About The Open Source Initiative
Founded in 1998, the Open Source Initiative (OSI) protects and promotes open source software, development and communities, championing software freedom in society through education, collaboration, and infrastructure, stewarding the Open Source Definition, and preventing abuse of the ideals and ethos inherent to the open source movement. The OSI is a public charity with global vision based in California. For more information about the OSI, please see https://opensource.org.