FLOSS Project Planets

LibrePlanet 2020 needs you: Volunteer today!

FSF Blogs - Tue, 2020-01-28 11:27

The LibrePlanet 2020 conference is coming very soon, on March 14 and 15 at the Back Bay Events Center in Boston, and WE NEED YOU to make the world's premier gathering of free software enthusiasts a success.

Volunteers are needed for several different tasks at LibrePlanet, from an audio/visual crew to point cameras and adjust microphones, to room monitors to introduce speakers, to a set-up and clean-up crew to make our conference appear and disappear at the Event Center, and more! You can volunteer for as much or as little time as you like, whether you choose to help out for an hour or two, or the entirety of both days. Either way, we'll provide you with a VERY handsome LibrePlanet 2020 shirt in your size, in addition to free admission to the entire conference and lunch and our eternal gratitude.

Excited? If you're ready to help put on an excellent conference, we are more than ready to show you how. One important step is to come to an in-person training and info session at the Free Software Foundation office, in downtown Boston. We have scheduled six training sessions beginning late February; the last one is the afternoon of the day immediately before LibrePlanet, which is perfect for people arriving from far away for the event. Please come to one if you can! Some volunteer tasks (room monitors, A/V crew) require more training than others, but there are some important things we need all volunteers to know, and attending a training will ensure that you're fully informed. The schedule for trainings is at the bottom of this email.

You're interested? Wonderful. Please reply to this email or write to resources@fsf.org. Let me know your T-shirt size (we'll have unisex S-XXXXL and fitted S-XXXL) and which training you can make it to. You can certainly volunteer without making it to a training -- I'll send you some info via email -- but your role may be a little less glamorous. Please also feel free to contact me with any questions or suggestions you may have; I will respond eagerly to your queries.

THANK YOU for supporting the Free Software Foundation and THANK YOU for volunteering for an excellent LibrePlanet!

LIBREPLANET 2020 VOLUNTEER TRAINING & INFO SESSION DATES:

All except one of these take place from 6 PM to 8 PM at the FSF office, 51 Franklin Street, Fifth floor, Downtown Crossing, Boston:

  • Wednesday, February 19
  • Tuesday, February 25
  • Thursday, February 27 (includes A/V training)
  • Wednesday, March 4 (includes A/V training
  • Tuesday, March 10 (includes A/V training
  • Friday, March 13: This is an afternoon session for people coming to town late, starting at 3 PM! It will also be at the FSF office, prior to the Friday night open house.
Categories: FLOSS Project Planets

Tag1 Consulting: Modern testing: part 3

Planet Drupal - Tue, 2020-01-28 11:14
Testing is becoming an essential keyword and toolkit for developers and development teams who seek to architect and implement successful and performant websites. Thanks to the unprecedented growth in automated testing tools and continuous integration (CI) solutions for all manner of web projects, testing is now table stakes for any implementation. That said, many developers find automated testing to be an altogether intimidating area of exploration. Fortunately, when paired with a development culture that values quality assurance (QA), you can focus on adding business value instead of fixing issues day in and day out. Three years ago, Yuriy Gerasimov (Senior Back-End Engineer at Tag1 Consulting ) gave a talk at DrupalCon New Orleans about some of the key ideas that Drupal developers need to understand in order to implement robust testing infrastructures and to foster a testing-oriented development culture that yields unforeseen business dividends across a range of projects. In this four-part blog series, we summarize some of the most important conclusions from Yuriy’s talk. And in this third installment, we’ll take a closer look at two of the most essential parts of any testing toolkit: unit testing and functional testing. Unit testing Unit testing is a particularly fascinating topic,... Read more preston Tue, 01/28/2020 - 13:59
Categories: FLOSS Project Planets

PyCharm: Webinar Recording: “Advanced Debugging in PyCharm”

Planet Python - Tue, 2020-01-28 11:00

Last week we held a special webinar for “Advanced Debugging in PyCharm”. Special how? In person, in the St. Petersburg office, with the two PyCharm team members in charge of the debugger, and a huge webinar audience. The recording is now available.

In this webinar, Liza Shashkova covered a long list of intermediate debugger tips and features, done in the context of writing a Tetris game using Arcade. Quite a number of really useful features, including some that even the pros might not know about.

Andrey Lisin did one section on remote debugging in PyCharm Professional, followed by a series of slides on the architecture of debuggers. This came from an internal talk Liza and he gave to the team when we were planning upcoming features.

Liza has a repo for her part and Andrey’s material is also available.

We wound up with a big turnout of attendees with a bunch of good questions: just what we were hoping for.

Categories: FLOSS Project Planets

Acro Media: Drupal as Middleware: How to Patch Digital Gaps in a Growing Business

Planet Drupal - Tue, 2020-01-28 10:45

Due to the nature of B2B sales, one of my roles is cold outreach. Most of the time my first method of outreach garners no replies. However, every so often I will receive a prompt email message or reply over the phone. It usually goes something along the lines of: “We already have a web development agency.” or “We are not interested.” I often wish I was at a sit-down meeting when these situations arise. This is because I simply cannot describe the multi-faceted solutions Drupal can provide, far and above a typical web development agency. “You should absolutely stay with them” is my typical response to prospects that have an agency they are happy with. I say this because there is so much more Drupal, as a business solution, can provide without even interacting with the frontend of their website. What we often promote with Drupal is its capability to create a more complete digital experience platform (DXP), not just a website.

What Gartner has to say about the DXP

In a 2019 report, Gartner has this analysis about DXPs:

“Driven by digital transformation efforts, organizations are moving beyond the traditional audience engagement resources of websites and mobile apps. There is a growing acceptance of the idea of digital experience platforms as vital to these efforts. DXPs provide a scalable foundation for creating, optimizing, integrating, delivering and managing a range of digital experiences for different audiences, both internal and external.1”

So let me unpack that a little bit in my own words. Essentially, your website and mobile apps are still very much at the forefront of digital marketing. Moving forward, though, more organizations have and will continue to create a more cohesive, single platform (DXP) in order to cater to all stakeholders of the company. This not only includes said organizations’ customers but also their teams and employees, allowing for a more comprehensive snapshot of the company from the outside and inside. In the same report, Gartner estimates that:

“Through 2021, 85% of the effort and cost involved in a digital experience platform (DXP) program will be spent on integrations with internal and external systems, including the DXP’s own, built-in capabilities.1”

In my opinion, this assumption by Gartner indicates that organizations are already well aware of the advantages a DXP can provide. If you're interested, click on the banner below to read the report.

An imaginary business case for a DXP  

Imagine you started a business selling gadgets. Your gadgets were better for target market X because they were less complicated than the gadgets that were available on the market at the time. So first off, you rented a storefront and sold the gadgets in your store, but you also realized you could scale your business by selling the same gadgets online. So in addition to your point of sale system (POS), you now had to adopt an appropriate ecommerce platform and build a website to sell the gadgets online.

Now that you were selling gadgets online you also had to have a shipping channel and a returns channel for replacing defective gadgets. As demand for your product began to grow you needed more gadgets on hand at any given time. The obvious solution was a warehouse, but you also needed a product information management (PIM) system to keep on top of your inventory and distribution channels.

As your product created a name for itself you opened a few more gadget stores and to satisfy demand across the globe you began selling your product through Amazon. With increased demand came competition, so in response, you invested in marketing software to stay on top of the industry trends. You began to diversify in order to make your business more resilient to market volatility. The diversification led to custom gadgets in addition to a gadget service and repair business.

In order to keep track of what your customers had purchased and to identify opportunities for cross-selling and up-selling, you invested in a customer relationship management (CRM) system. Finally, just under a year ago you invested in a new enterprise resource planning (ERP) system. This way all of your new departments had the information they needed to operate efficiently.

So we are now in the present day. Like many other businesses that grew at a rapid pace, you find yourself in a situation where all of your technology has become siloed. In this analogy, your data and information all exist, but it is locked away in separate silos. Each silo represents a piece of software, a distribution channel, a legacy POS system, or that missing Amazon integration that can only be accessed from one place. You can run a business this way, and many organizations do just that, without realizing that there is actually a more efficient way to do things. This is where the DXP comes into play. What you would prefer rather than individual silos would be a horizontal technology architecture with open lines of communication between everything. This, as one could imagine, can save a tremendous amount of time and manual workflows, eliminating what we call swivel-chair processes. Simply stated it is a more efficient way of doing business. The problem is many business owners and decision makers may not even realize this is happening because they live in their own silos and no one has pointed it out to them.

How does Drupal come into play?

Drupal is a content management system (CMS), but at the same time, Drupal can do so much more than a traditional CMS. Through API integrations, also known as API hooks, Drupal has the ability to be used as middleware. As middleware, Drupal can act as a modular engine that connects all the data from the aforementioned gadget business’ technology. Data can flow forwards and backward between the various pieces of technology and even integrate into legacy systems like the POS in the gadget example. Furthermore, the modular nature of Drupal middleware essentially future proofs your business allowing for business scalability.

To give an analogy, you can think of Drupal middleware as a computer with unlimited USB ports. The computer acts as the brain passing information back and forth between systems and the USB ports are the API hooks. USB ports are non-proprietary and you can, therefore, unplug cables you no longer need and replace them with new cables. You can also add or remove cables as necessary and the computer keeps on functioning as long as you configure the drivers. So as you outgrow software systems or you decide to replace that legacy POS, no problem because you can just plug in the new software, install the drivers, and you are back up and running again.

Connecting it all together

So to return to my statement at the beginning of this blog post, the reason I wish I could sit down with those who respond so quickly to my first cold outreach is that I do not want to be typecast as just another web development agency. In actual fact, because our expertise lies in Drupal we are far better positioned to provide solutions that lay beyond the scope of the traditional idea of a website. Sure we can develop an incredibly robust frontend experience and, likewise, a flexible scalable ecommerce engine. But, we can also use Drupal as middleware that allows for seamless flow of information between systems.

If you've read this and would like to have a quick chat, let us know! We're happy to help. I also mentioned a Gartner report from 2019 that is a great introduction for anyone trying to nail down their digital experience platform. Gartner has made this report available to us to share with our readers for a limited time, so check it out now while you still can.

1 - Gartner, "Don’t Wreck Digital Engagement With Bad Deployment Decisions for Your Digital Experience Platform,” 31 July 2019, Gene Phifer.

Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Categories: FLOSS Project Planets

Real Python: Python Modules and Packages: An Introduction

Planet Python - Tue, 2020-01-28 09:00

In this course, you’ll learn about Python modules and Python packages, two mechanisms that facilitate modular programming.

Modular programming is the process of breaking a large, unwieldy programming task into separate, smaller, more manageable subtasks or modules. Individual modules can then be put together like building blocks to create a larger application.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Stack Abuse: Text Classification with BERT Tokenizer and TF 2.0 in Python

Planet Python - Tue, 2020-01-28 08:46

This is the 23rd article in my series of articles on Python for NLP. In the previous article of this series, I explained how to perform neural machine translation using seq2seq architecture with Python's Keras library for deep learning.

In this article we will study BERT, which stands for Bidirectional Encoder Representations from Transformers and its application to text classification. BERT is a text representation technique like Word Embeddings. If you have no idea of how word embeddings work, take a look at my article on word embeddings.

Like word embeddings, BERT is also a text representation technique which is a fusion of variety of state-of-the-art deep learning algorithms, such as bidirectional encoder LSTM and Transformers. BERT was developed by researchers at Google in 2018 and has been proven to be state-of-the-art for a variety of natural language processing tasks such text classification, text summarization, text generation, etc. Just recently, Google announced that BERT is being used as a core part of their search algorithm to better understand queries.

In this article we will not go into the mathematical details of how BERT is implemented, as there are plenty of resources already available online. Rather we will see how to perform text classification using the BERT Tokenizer. In this article you will see how the BERT Tokenizer can be used to create text classification model. In the next article I will explain how the BERT Tokenizer, along with BERT embedding layer, can be used to create even more efficient NLP models.

Note: All the scripts in this article have been tested using Google Colab environment, with Python runtime set to GPU.

The Dataset

The dataset used in this article can be downloaded from this Kaggle link.

If you download the dataset and extract the compressed file, you will see a CSV file. The file contains 50,000 records and two columns: review and sentiment. The review column contains text for the review and the sentiment column contains sentiment for the review. The sentiment column can have two values i.e. "positive" and "negative" which makes our problem a binary classification problem.

We have previously performed sentimental analysis of this dataset in a previous article where we achieved maximum accuracy of 92% on the training set via word a embedding technique and convolutional neural network. On the test set the maximum accuracy achieved was 85.40% using the word embedding and single LSTM with 128 nodes. Let's see if we can get better accuracy using BERT representation.

Installing and Importing Required Libraries

Before you can go and use the BERT text representation, you need to install BERT for TensorFlow 2.0. Execute the following pip commands on your terminal to install BERT for TensorFlow 2.0.

!pip install bert-for-tf2 !pip install sentencepiece

Next, you need to make sure that you are running TensorFlow 2.0. Google Colab, by default, doesn't run your script on TensorFlow 2.0. Therefore, to make sure that you are running your script via TensorFlow 2.0, execute the following script:

try: %tensorflow_version 2.x except Exception: pass import tensorflow as tf import tensorflow_hub as hub from tensorflow.keras import layers import bert

In the above script, in addition to TensorFlow 2.0, we also import tensorflow_hub, which basically is a place where you can find all the prebuilt and pretrained models developed in TensorFlow. We will be importing and using a built-in BERT model from TF hub. Finally, if in the output you see the following output, you are good to go:

TensorFlow 2.x selected. Importing and Preprocessing the Dataset

The following script imports the dataset using the read_csv() method of the Pandas dataframe. The script also prints the shape of the dataset.

movie_reviews = pd.read_csv("/content/drive/My Drive/Colab Datasets/IMDB Dataset.csv") movie_reviews.isnull().values.any() movie_reviews.shape

Output

(50000, 2)

The output shows that our dataset has 50,000 rows and 2 columns.

Next, we will preprocess our data to remove any punctuations and special characters. To do so, we will define a function that takes as input a raw text review and returns the corresponding cleaned text review.

def preprocess_text(sen): # Removing html tags sentence = remove_tags(sen) # Remove punctuations and numbers sentence = re.sub('[^a-zA-Z]', ' ', sentence) # Single character removal sentence = re.sub(r"\s+[a-zA-Z]\s+", ' ', sentence) # Removing multiple spaces sentence = re.sub(r'\s+', ' ', sentence) return sentence TAG_RE = re.compile(r'<[^>]+>') def remove_tags(text): return TAG_RE.sub('', text)

The following script cleans all the text reviews:

reviews = [] sentences = list(movie_reviews['review']) for sen in sentences: reviews.append(preprocess_text(sen))

Our dataset contains two columns, as can be verified from the following script:

print(movie_reviews.columns.values)

Output:

['review' 'sentiment']

The review column contains text while the sentiment column contains sentiments. The sentiments column contains values in the form of text. The following script displays unique values in the sentiment column:

movie_reviews.sentiment.unique()

Output:

array(['positive', 'negative'], dtype=object)

You can see that the sentiment column contains two unique values i.e. positive and negative. Deep learning algorithms work with numbers. Since we have only two unique values in the output, we can convert them into 1 and 0. The following script replaces positive sentiment by 1 and the negative sentiment by 0.

y = movie_reviews['sentiment'] y = np.array(list(map(lambda x: 1 if x=="positive" else 0, y)))

Now the reviews variable contain text reviews while the y variable contains the corresponding labels. Let's randomly print a review.

print(reviews[10])

Output:

Phil the Alien is one of those quirky films where the humour is based around the oddness of everything rather than actual punchlines At first it was very odd and pretty funny but as the movie progressed didn find the jokes or oddness funny anymore Its low budget film thats never problem in itself there were some pretty interesting characters but eventually just lost interest imagine this film would appeal to stoner who is currently partaking For something similar but better try Brother from another planet

It clearly looks like a negative review. Let's just confirm it by printing the corresponding label value:

print(y[10])

Output:

0

The output 0 confirms that it is a negative review. We have now preprocessed our data and we are now ready to create BERT representations from our text data.

Creating a BERT Tokenizer

In order to use BERT text embeddings as input to train text classification model, we need to tokenize our text reviews. Tokenization refers to dividing a sentence into individual words. To tokenize our text, we will be using the BERT tokenizer. Look at the following script:

BertTokenizer = bert.bert_tokenization.FullTokenizer bert_layer = hub.KerasLayer("https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/1", trainable=False) vocabulary_file = bert_layer.resolved_object.vocab_file.asset_path.numpy() to_lower_case = bert_layer.resolved_object.do_lower_case.numpy() tokenizer = BertTokenizer(vocabulary_file, to_lower_case)

In the script above we first create an object of the FullTokenizer class from the bert.bert_tokenization module. Next, we create a BERT embedding layer by importing the BERT model from hub.KerasLayer. The trainable parameter is set to False, which means that we will not be training the BERT embedding. In the next line, we create a BERT vocabulary file in the form a numpy array. We then set the text to lowercase and finally we pass our vocabulary_file and to_lower_case variables to the BertTokenizer object.

It is pertinent to mention that in this article, we will only be using BERT Tokenizer. In the next article we will use BERT Embeddings along with tokenizer.

Let's now see if our BERT tokenizer is actually working. To do so, we will tokenize a random sentence, as shown below:

tokenizer.tokenize("don't be so judgmental")

Output:

['don', "'", 't', 'be', 'so', 'judgment', '##al']

You can see that the text has been successfully tokenized. You can also get the ids of the tokens using the convert_tokens_to_ids() of the tokenizer object. Look at the following script:

tokenizer.convert_tokens_to_ids(tokenizer.tokenize("dont be so judgmental"))

Output:

[2123, 2102, 2022, 2061, 8689, 2389]

Now will define a function that accepts a single text review and returns the ids of the tokenized words in the review. Execute the following script:

def tokenize_reviews(text_reviews): return tokenizer.convert_tokens_to_ids(tokenizer.tokenize(text_reviews))

And execute the following script to actually tokenize all the reviews in the input dataset:

tokenized_reviews = [tokenize_reviews(review) for review in reviews] Prerparing Data For Training

The reviews in our dataset have varying lengths. Some reviews are very small while others are very long. To train the model, the input sentences should be of equal length. To create sentences of equal length, one way is to pad the shorter sentences by 0s. However, this can result in a sparse matrix contain large number of 0s. The other way is to pad sentences within each batch. Since we will be training the model in batches, we can pad the sentences within the training batch locally depending upon the length of the longest sentence. To do so, we first need to find the length of each sentence.

The following script creates a list of lists where each sublist contains tokenized review, the label of the review and the length of the review:

reviews_with_len = [[review, y[i], len(review)] for i, review in enumerate(tokenized_reviews)]

In our dataset, the first half of the reviews are positive while the last half contains negative reviews. Therefore, in order to have both positive and negative reviews in the training batches we need to shuffle the reviews. The following script shuffles the data randomly:

random.shuffle(reviews_with_len)

Once the data is shuffled, we will sort the data by the length of the reviews. To do so, we will use the sort() function of the list and will tell it that we want to sort the list with respect to the third item in the sublist i.e. the length of the review.

reviews_with_len.sort(key=lambda x: x[2])

Once the reviews are sorted by length, we can remove the length attribute from all the reviews. Execute the following script to do so:

sorted_reviews_labels = [(review_lab[0], review_lab[1]) for review_lab in reviews_with_len]

Once the reviews are sorted we will convert thed dataset so that it can be used to train TensorFlow 2.0 models. Run the following code to convert the sorted dataset into a TensorFlow 2.0-compliant input dataset shape.

processed_dataset = tf.data.Dataset.from_generator(lambda: sorted_reviews_labels, output_types=(tf.int32, tf.int32))

Finally, we can now pad our dataset for each batch. The batch size we are going to use is 32 which means that after processing 32 reviews, the weights of the neural network will be updated. To pad the reviews locally with respect to batches, execute the following:

BATCH_SIZE = 32 batched_dataset = processed_dataset.padded_batch(BATCH_SIZE, padded_shapes=((None, ), ()))

Let's print the first batch and see how padding has been applied to it:

next(iter(batched_dataset))

Output:

(<tf.Tensor: shape=(32, 21), dtype=int32, numpy= array([[ 2054, 5896, 2054, 2466, 2054, 6752, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 3078, 5436, 3078, 3257, 3532, 7613, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 3191, 1996, 2338, 5293, 1996, 3185, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 2062, 23873, 3993, 2062, 11259, 2172, 2172, 2062, 14888, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 1045, 2876, 9278, 2023, 2028, 2130, 2006, 7922, 12635, 2305, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], ...... [ 7244, 2092, 2856, 10828, 1997, 10904, 2402, 2472, 3135, 2293, 2466, 2007, 10958, 8428, 10102, 1999, 1996, 4281, 4276, 3773, 0], [ 2005, 5760, 7788, 4393, 8808, 2498, 2064, 12826, 2000, 1996, 11056, 3152, 3811, 16755, 2169, 1998, 2296, 2028, 1997, 2068, 0], [ 2307, 3185, 2926, 1996, 2189, 3802, 2696, 2508, 2012, 2197, 2023, 8847, 6702, 2043, 2017, 2031, 2633, 2179, 2008, 2569, 2619], [ 2028, 1997, 1996, 4569, 15580, 2102, 5691, 2081, 1999, 3522, 2086, 2204, 23191, 5436, 1998, 11813, 6370, 2191, 2023, 2028, 4438], [ 2023, 3185, 2097, 2467, 2022, 5934, 1998, 3185, 4438, 2004, 2146, 2004, 2045, 2024, 2145, 2111, 2040, 6170, 3153, 1998, 2552]], dtype=int32)>, <tf.Tensor: shape=(32,), dtype=int32, numpy= array([0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1], dtype=int32)>)

The above output shows the first five and last five padded reviews. From the last five reviews, you can see that the total number of words in the largest sentence were 21. Therefore, in the first five reviews the 0s are added at the end of the sentences so that their total length is also 21. The padding for the next batch will be different depending upon the size of the largest sentence in the batch.

Once we have applied padding to our dataset, the next step is to divide the dataset into test and training sets. We can do that with the help of following code:

TOTAL_BATCHES = math.ceil(len(sorted_reviews_labels) / BATCH_SIZE) TEST_BATCHES = TOTAL_BATCHES // 10 batched_dataset.shuffle(TOTAL_BATCHES) test_data = batched_dataset.take(TEST_BATCHES) train_data = batched_dataset.skip(TEST_BATCHES)

In the code above we first find the total number of batches by dividing the total records by 32. Next, 10% of the data is left aside for testing. To do so, we use the take() method of batched_dataset() object to store 10% of the data in the test_data variable. The remaining data is stored in the train_data object for training using the skip() method.

The dataset has been prepared and now we are ready to create our text classification model.

Creating the Model

Now we are all set to create our model. To do so, we will create a class named TEXT_MODEL that inherits from the tf.keras.Model class. Inside the class we will define our model layers. Our model will consist of three convolutional neural network layers. You can use LSTM layers instead and can also increase or decrease the number of layers. I have copied the number and types of layers from SuperDataScience's Google colab notebook and this architecture seems to work quite well for the IMDB Movie reviews dataset as well.

Let's now create out model class:

class TEXT_MODEL(tf.keras.Model): def __init__(self, vocabulary_size, embedding_dimensions=128, cnn_filters=50, dnn_units=512, model_output_classes=2, dropout_rate=0.1, training=False, name="text_model"): super(TEXT_MODEL, self).__init__(name=name) self.embedding = layers.Embedding(vocabulary_size, embedding_dimensions) self.cnn_layer1 = layers.Conv1D(filters=cnn_filters, kernel_size=2, padding="valid", activation="relu") self.cnn_layer2 = layers.Conv1D(filters=cnn_filters, kernel_size=3, padding="valid", activation="relu") self.cnn_layer3 = layers.Conv1D(filters=cnn_filters, kernel_size=4, padding="valid", activation="relu") self.pool = layers.GlobalMaxPool1D() self.dense_1 = layers.Dense(units=dnn_units, activation="relu") self.dropout = layers.Dropout(rate=dropout_rate) if model_output_classes == 2: self.last_dense = layers.Dense(units=1, activation="sigmoid") else: self.last_dense = layers.Dense(units=model_output_classes, activation="softmax") def call(self, inputs, training): l = self.embedding(inputs) l_1 = self.cnn_layer1(l) l_1 = self.pool(l_1) l_2 = self.cnn_layer2(l) l_2 = self.pool(l_2) l_3 = self.cnn_layer3(l) l_3 = self.pool(l_3) concatenated = tf.concat([l_1, l_2, l_3], axis=-1) # (batch_size, 3 * cnn_filters) concatenated = self.dense_1(concatenated) concatenated = self.dropout(concatenated, training) model_output = self.last_dense(concatenated) return model_output

The above script is pretty straightforward. In the constructor of the class, we initialze some attributes with default values. These values will be replaced later on by the values passed when the object of the TEXT_MODEL class is created.

Next, three convolutional neural network layers have been initialized with the kernel or filter values of 2, 3, and 4, respectively. Again, you can change the filter sizes if you want.

Next, inside the call() function, global max pooling is applied to the output of each of the convolutional neural network layer. Finally, the three convolutional neural network layers are concatenated together and their output is fed to the first densely connected neural network. The second densely connected neural network is used to predict the output sentiment since it only contains 2 classes. In case you have more classes in the output, you can updated the output_classes variable accordingly.

Let's now define the values for the hyper parameters of our model.

VOCAB_LENGTH = len(tokenizer.vocab) EMB_DIM = 200 CNN_FILTERS = 100 DNN_UNITS = 256 OUTPUT_CLASSES = 2 DROPOUT_RATE = 0.2 NB_EPOCHS = 5

Next, we need to create an object of the TEXT_MODEL class and pass the hyper paramters values that we defined in the last step to the constructor of the TEXT_MODEL class.

text_model = TEXT_MODEL(vocabulary_size=VOCAB_LENGTH, embedding_dimensions=EMB_DIM, cnn_filters=CNN_FILTERS, dnn_units=DNN_UNITS, model_output_classes=OUTPUT_CLASSES, dropout_rate=DROPOUT_RATE)

Before we can actually train the model we need to compile it. The following script compiles the model:

if OUTPUT_CLASSES == 2: text_model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"]) else: text_model.compile(loss="sparse_categorical_crossentropy", optimizer="adam", metrics=["sparse_categorical_accuracy"])

Finally to train our model, we can use the fit method of the model class.

text_model.fit(train_data, epochs=NB_EPOCHS)

Here is the result after 5 epochs:

Epoch 1/5 1407/1407 [==============================] - 381s 271ms/step - loss: 0.3037 - accuracy: 0.8661 Epoch 2/5 1407/1407 [==============================] - 381s 271ms/step - loss: 0.1341 - accuracy: 0.9521 Epoch 3/5 1407/1407 [==============================] - 383s 272ms/step - loss: 0.0732 - accuracy: 0.9742 Epoch 4/5 1407/1407 [==============================] - 381s 271ms/step - loss: 0.0376 - accuracy: 0.9865 Epoch 5/5 1407/1407 [==============================] - 383s 272ms/step - loss: 0.0193 - accuracy: 0.9931 <tensorflow.python.keras.callbacks.History at 0x7f5f65690048>

You can see that we got an accuracy of 99.31% on the training set.

Let's now evaluate our model's performance on the test set:

results = text_model.evaluate(test_dataset) print(results)

Output:

156/Unknown - 4s 28ms/step - loss: 0.4428 - accuracy: 0.8926[0.442786190037926, 0.8926282]

From the output, we can see that we got an accuracy of 89.26% on the test set.

Conclusion

In this article you saw how we can use BERT Tokenizer to create word embeddings that can be used to perform text classification. We performed sentimental analysis of IMDB movie reviews and achieved an accuracy of 89.26% on the test set. In this article we did not use BERT embeddings, we only used BERT Tokenizer to tokenize the words. In the next article, you will see how BERT Tokenizer along with BERT Embeddings can be used to perform text classification.

Categories: FLOSS Project Planets

Specbee: Implementing Google Directions Module in Drupal 8 – A quick guide

Planet Drupal - Tue, 2020-01-28 07:38
Implementing Google Directions Module in Drupal 8 – A quick guide Ankitha 28 Jan, 2020 Top 10 best practices for designing a perfect UX for your mobile app

Communicating with Google services is simple and effortless with a wealth of API’s to choose from. With Drupal 8’s powerful API-first capabilities, interacting with these services are now easier and more effective. The API in focus is the Google Directions API. Drupal 8 offers a Google Directions module that integrates with the Google Directions API. 

What does the Google Directions Module do?

The Drupal Google Directions module allows you to create and add a block to your content where users can add two locations and find directions between the two points. Also displayed are suggested routes based on modes of transport (public transit, cycling, walking or driving) and the time it will take to get there. This block can be positioned via the Block layout and customized to fit any website’s style guidelines.

Getting Started with Google Directions Module

Let’s get started with implementing the Google Directions module.

Get the Google API Key 

A Google API Key is required to access the Google Directions Service.
To get a key, visit https://developers.google.com/maps/documentation/directions/start with your Google account and follow the Guide.
To get an API key:

  • Visit the Google Cloud Platform Console.
  • Click on the project drop-down and select or create the project for which you want to add an API key.
     

     

  • Click on the menu button and select APIs & Services  -> Credentials
  • On the Credentials page, click on Create credentials -> API key.
  • The “API key created” dialog displays your newly created API key. Copy the API key as it must be configured in your Drupal Project.

     

  •   Select Close. The new API key is listed on the Credentials page under API keys.

     

Enable the required APIs
  • In the Google Developer Console Dashboard, click on the menu button, select APIs & Services -> Library

     

  • Search for and enable the following APIs

                   ○ Google Directions API

                   ○ Google Maps JavaScript API

                   ○ Google Places API

                   ○ Google Geocoding API

Installing

Install and enable the Google Directions module as you would normally install a contributed Drupal module. Use your preferred method to download the modules. I’m using the Composer to install since it automatically takes care of all the necessary dependencies and Drush command to enable the module.

specbee@specbee-HP-ProBook-640-G1:/var/www/html/drupal_php$ composer require drupal/google_directions specbee@specbee-HP-ProBook-640-G1:/var/www/html/drupal_php$ drush en google_directions
Configuration
  • Configure user permissions as required in Administration -> People -> Permissions:       
               ○  Administer Google Directions

     

  • Once Google Directions is installed and enabled and permissions are granted:
               ○ Visit /admin/config/services/google-directions to enter your Google API Key that we copied earlier.
  • ○ Use the Block layout page (/admin/structure/block) to position the block in one or more regions to suit your use case for the Google Directions module.
  •             ○ For each block placement, configure Visibility and other block settings to suit your use case.
                ○ Click on Save Block and visit the page. The following Google Directions block appears in the region it was placed.

Good to Note : 


If the menu does not display, flush cache and try again.

  • In the block enter the locations in “From” and “To” fields and hit “Go”.

 

The API-first initiative taken up by the community in Drupal 8 is giving a lot of power and flexibility in the hands of developers and website owners. Talking to API’s like the above discussed Google Directions API via the Drupal 8 Google Directions module is easy and compelling. Need help with setting up your Drupal website and leveraging Drupal 8’s power? Learn more about our Drupal services and get it touch with us.  

Drupal Planet Shefali ShettyApr 05, 2017 Subscribe For Our Newsletter And Stay Updated Subscribe

Leave us a Comment

  Shefali ShettyApr 05, 2017 Recent Posts Image Implementing Google Directions Module in Drupal 8 – A quick guide Image A Prelaunch Checklist for your Drupal Website Image Cryptocurrency and Blockchain - Everything You Need to Know Want to extract the maximum out of Drupal? Contact us TALK TO US Featured Success Stories

Know more about our technology driven approach to recreate the content management workflow for [24]7.ai

link

Find out how we transformed the digital image of world’s largest healthcare provider, an attribute that defined their global presence in the medical world.

link

Discover how a Drupal powered internal portal encouraged the sellers at Flipkart to obtain the latest insights with respect to a particular domain.

link
Categories: FLOSS Project Planets

recutils @ Savannah: Pre-release 1.8.90 in alpha.gnu.org

GNU Planet! - Tue, 2020-01-28 06:31

The pre-release recutils-1.8.90.tar.gz is now available at ftp://alpha.gnu.org/gnu/recutils/recutils-1.8.90.tar.gz

The NEWS file in the tarball contains a list of the changes since 1.8.

The planned date for releasing 1.9 is Saturday 1 February 2020.

Please report any problem found with the pre-release to bug-recutils@gnu.org.

Thanks!

Categories: FLOSS Project Planets

ComputerMinds.co.uk: Sponsor work to improve Drupal for us all

Planet Drupal - Tue, 2020-01-28 06:01

We love making great websites, standing on the shoulders of the giants that have gone before us to make Drupal what it is today. Thousands of working hours from fantastic people around the world have gone into producing this digital experience platform. Many of those gave voluntarily for free, while many were directed by paying customers who gave back so the rest of us could benefit. Everyone stands to gain when innovation, graft and capacity are pooled together. All that functionality in your Drupal site has been written by a real person at some point, who considered it worth sharing with the world. That is only the tip of the iceberg too - there is an entire community behind Drupal and the ecosystem of contributed modules that can be added to it.

Do you recognise what you have in your site? It's easy to dwell on imperfections, but consider for a moment how every piece of your project has been produced by someone trying to make things possible, which weren't before. Would you like more of that? Would you like to help millions of users across the world do more too? All kinds of outfits use Drupal - non-profits, charities, educational institutions, public sector organisations as well as tiny companies and international enterprises. Here's an example of some amazing work another company has done with Drupal to help provide better health care in rural Rwanda:

Spent a whole day in rural Rwanda, watching - in action - an @elmlang web-app with a #Drupal backend we've built for @ihanganeproject.

No marketing landing pages, no sale funnels, no targeted ads. Just a web-app that helps provide a better health care to mothers and their babies pic.twitter.com/PnLqV0vpSQ

— Amitai Burstein (@amitaibu) November 14, 2019

It's incredible what can be done with Drupal! If we all seek to help improve it as a platform, rather than just taking from it, our websites & companies stand to gain, sure. But that way, we also allow those less fortunate than ourselves to benefit. If you already get a boost from what others have put into Drupal, could you give something back too?

We're proud of our regular habit of contributing fixes back to the Drupal modules that we use. We've been one of the top 30 organisations contributing to Drupal core itself for the last couple of years, and we've gone beyond code contributions to sponsor events and promote Drupal in our local communities. We've maintained major contrib projects, such as Aegir & Views data export. Wherever possible, we develop in a way that allows work to be re-used in future, hence our 'Doing Drupal right' tagline. We believe working for our clients and the community is the right way. 

You can help this mission, and the health of the Drupal project as a whole by joining with us. Whilst open source products may not cost anything, the work to create them does. It won't be sustainable, nor desirable, for Drupal only to be built by volunteers, or for funding to be left to 'everyone else'. You can even help shape the direction of Drupal by getting involved.

We very recently completed a release of the Language Hierarchy module for Drupal 8, because Julabo GmbH got in touch to sponsor that work after recognising our contribution to that project and our multilingual expertise. They needed to add functionality and expedite work on that project to use it on their own site. One of our own clients already used that project too ... now the rest of the Drupal community can benefit from what they have facilitated!

We would love for you to sponsor work on Drupal or contributed modules. Speak to your developer about this. We're making ourselves available for sponsored work for the community. Maybe your site relies on modules contributed by the community which don't quite work as you'd like - so we could fix that for you and everyone else out there who may have similar needs. Maybe your project has bespoke functionality that others could use if it were generalised? Sponsoring that kind of work is an opportunity to make it more robust for you too, and then others around the world may even pick it up to add improvements that you haven't even dreamed of yet. Perhaps we can help you meet accessibility requirements or accomplish those 'nice-to-have' features that you've had on a backlog, by expanding what Drupal's modules can do. Maybe that even qualifies as something you can use a community or R&D budget for!

Would you like to expand what Drupal and therefore your own website can do, whilst sharing that with the thousands of users who may never be in a position to dream of building such functionality? Get in touch with us to discuss sponsoring Drupal work. We can suggest specific areas needing work, or you might want to improve something closer to your heart. Over to you.

 

Photo by Helena Lopes on Unsplash

Categories: FLOSS Project Planets

Interview with Spihon

Planet KDE - Tue, 2020-01-28 05:48
Could you tell us something about yourself?

Hi, I’m Spihon, I’m a born Desert Rat of the United States who loves to read, dabble in a bit of writing and play tabletop games from D&D to Palladium campaigns.

And a heavy user of nom de plumes. I’m still nervous about putting my real name on my art, so like one of my other pen names I’m an enygma.

Oh and I am a proud member of the 501st Legion for Charity, who do walks for causes like hemophilia and autism.

Do you paint professionally, as a hobby artist, or both?

Well to be honest I would love to draw professionally and making it a living, so right now it’s a hobby that I take seriously, but hopefully I can make it work out in the end.

What genre(s) do you work in?

O_O…^^;

Boy that’s a broad section to cover, but I think I can narrow it down to sci-fi and fantasy with a bit of abstract surrealism (I honestly need to try that in Krita later). Anyway those are some of the things I feel comfortable working in.

Whose work inspires you most — who are your role models as an artist?

Again another tough question, I actually have several people who inspired me. One of them is Don Bluth and one of his employees, Dermot O’Connor, who has been a huge help in my art growth. And also Mike Crilley. Of the others I don’t personally know their names so it’s going to be their handles (which you can find on DeviantArt): my best friend Gothicraft, Rottenribcage, Onixx, Ipku, Taleea, Zarla, cgratzlaff, AbsoluteDream.

And finally my family: my cousins, Mrs. Battle, Stardewvalleycoconut, ToastyPaw, Nana, my aunt Nikki and my sister Sailorstar237, my uncles JGriz and Vraptor, and last but not least my mother. It’s because of these people that I got inspired to even dabble in art and also, without my family’s support, I don’t think I would have made it this far.

How and when did you get to try digital painting for the first time?

When I started digital painting — it makes me feel old but I think the first time was when I was about seven when I tried MS Paint for the first time, it was the only thing I had available as a kid and back then I wasn’t as good at it as I am now.

What makes you choose digital over traditional painting?

Well, for one thing digital is a bit more forgiving than traditional (well, if it doesn’t crash on you when you’ve been working on a piece for hours in it). And also money since I can’t keep replacing art supplies all the time.

Also I’ll be honest, I’m a lot neater with digital compared to traditional. Now don’t get me wrong: I love doing traditional, but someone who walked around with half their art project almost everywhere on their body…I guess you can get the picture. I get literally into my art.

How did you find out about Krita?

That’s an easy one, Which ties in with digital… money. About 2018 I was busy looking for a free art program that I could animate with, since I’m struggling with trying to find a job, so I thought I could do try my hand at making videos for YouTube. And speaking of YouTube, that’s where I found it, from this guy’s video on how to animate, and I was sold so I downloaded it and I’m not going back on it.

Actually, the anniversary of when I found it is next month, February 18th, so I’ll have been using it for two years.

What was your first impression?

Truthfully a bit intimidating at first, until I got the hang of it and it became my go to art program for everything I do, from simple paintings to comics. Heck, David Revoy even got me inspired to do it… Sure, I could have added him to the “who inspires me” section but come on! He needs a special place as my Krita Rockstar…

Anyhoo, I draw more these days than I play video games.

What do you love about Krita?

It’s a free professional program, I don’t have to worry about paying monthly subscription fees, and it’s easy to handle.

Also the continuous line tool and the auto smooth curve option, those make my line work a lot nicer.

What do you think needs improvement in Krita? Is there anything that really annoys you?

Well, as it is still in development I can’t complain much but… I guess the old fonts, I was using one of the original fonts that came with Krita at the time, and I can’t seem to find it again.

The similar color selection tool doesn’t like to be used much or I’m still lacking knowledge in it… Which is more than likely with me.

Now that’s just the stuff that annoys me. As for improvements, I think the gradients could have variety in changing opacity and a bit more control in direction when using it.

But other than that, it’s an amazing program.

What sets Krita apart from the other tools that you use?

Compared to my first experience of using MS Paint or Gimp, it’s a lot more user friendly (to me, that is). I can easily bind keys to my keyboard instead of struggling to have it stay in the art program itself.

If you had to pick one favorite of all your work done in Krita so far, what would it be, and why?

That’s a tough one to answer, since thanks to Krita, I’ve improved a lot in my art, but if I had to name one it would be my Jewel Thief PhantomWing. The one you’re seeing here in this interview is a redraw, since my sketch and my first attempt at drawing digital didn’t turn out the way I wanted so I challenged myself to attempt it with Krita, and I’m much happier with the results than before.

What techniques and brushes did you use in it?

If I can remember the steps and the brushes I used for it I’ll tell ya but I’m kind of of drawing a blank since when I did it was in the wee hours of the morning… And actually it was three days of that in sequence. I also have a lot of brushes in Krita right now…

So…Yeah what was the question again XD…^^;

My attempt at humor aside, the technique I used (if you can even call it that) is “splash colors on it and make it look pretty”. But to refine it I used the Bezier curve tool and my mouse to do the line work, the selection tools (except the similar color one), and the paint bucket. And my art tablet for any detail work.

Now as my first sketch was me just making the lines add up to make a picture, I used the reference image tool and used that to help finalize details for the final product. And lastly the gradient tool for easy shading.

As for the brushes, I mostly used the HC Concept brushes for the whole thing, so yeah it was literally that and the gradient tool.

Where can people see more of your work?

You can find me at my DeviantArt account, which is Spihon https://www.deviantart.com/spihon

To be honest I don’t do much social media, but you can find some of my current comic projects (some original, some fan) and a lot of photos of my earlier career as a 501st member when I didn’t have a bucket to speak of (which is my group’s slang for the helmets, if that helps).

Anything else you’d like to share?

Well, stay in school, and may good fortune be your ever constant companion!

 

Categories: FLOSS Project Planets

New features and changes in the Kuesa 1.1.0 release

Planet KDE - Tue, 2020-01-28 05:00

KUESA™ is a solution that provides an integrated and unified workflow for designers and developers to create, optimize and integrate real time 3D content in a 3D or hybrid 2D/3D software user interface. Models, including geometry, materials, animations and more, can smoothly be shared between designers and developers. Kuesa relies on glTF 2.0, an open standard which is being broadly used in the industry, so it can easily and seamlessly integrate with a growing number of authoring tools.

Kuesa is made of 3 main components:

  • In addition to supporting any compliant glTF 2.0 files, Kuesa provides plugins for content creation tools Blender and 3DS Max for creating glTF 2.0 files which support custom extensions.
  • A Qt module designed to load, render and manipulate glTF 2.0 models in applications using Qt 3D, making it easy to do things like triggering animations contained in the glTF files, finding camera details defined by the designer, etc. It also includes a library of post processing effects which can be easily extended.
  • Tools to help designers and developers inspect and optimise the models, including command line tools to automate the process

This blog post outlines the new features and changes in the 1.1.0 release.

The main objective for this release was to ensure compliance with the Khronos glTF 2.0 specification. We made sure that Kuesa works on a wide range of target hardware, be it desktop, mobile or embedded. In addition, a great deal of work has been put into a nice technical documentation, following the Qt standards. This documentation is available at the following address : https://kdab.github.io/kuesa/

Kuesa 1.1.0 supports Qt 5.12.5, 5.13, and will support Qt 5.14 when it is released.

Kuesa Ecosystem

This release is marked by an ever-growing ecosystem of Kuesa-related tools, bindings and integrations.

First and foremost, full Python bindings for Kuesa have been developed – learn more about this here. Support for Kuesa extensions, such as layer support, is being worked on in the official Khronos Blender glTF exporter ; we will keep you posted about the progress on that. The 3DS Max Kuesa exporter can on its side be downloaded here.

gltfEditor (ex-assetpipelineeditor) has been improved, and a simple, minimal-dependency glTF 2 viewer has been implemented. It supports a few command line flags useful for testing and is simply named gltfViewer.

A new tool to process glTF 2.0 assets from the command line has been implemented. It is named assetprocessor. It allows for instance to compress meshes, and embed buffers in a glTF file or extract them.

Breaking changes

Some tools have been renamed in order to make their main use case clearer.

  • assetpipelineeditor has been renamed gltfEditor.
  • The extension of shader graph save files has been changed from .qt3d to .graph (the files in kuesa/src/core/shaders/graphs).
  • The Material API has been refactored : all the properties on materials (MetallicRoughnessMaterial, UnlitMaterial) are now instead grouped in material-specific “Properties” class – MetallicRoughnessProperties, UnlitProperties. This allows to decouple the actual material logic from the parameters needed for rendering it.
New features

glTF 2.0 support has been greatly improved in this release. Morph targets, which were missing in Kuesa 1.0, are now available.

Support for a few extensions and niceties has been added :

  • KHR_lights_punctual : this feature manifests itself through the new Kuesa.DirectionalLight, Kuesa.PointLight and Kuesa.SpotLight objects.
  • KHR_materials_unlit : unlit materials are now also supported in glTF 2.0 files.
  • Support for loading GLB files (single-file binary glTF format) has landed
  • The GLTF2Importer now supports multiple glTF scenes.
  • Support for 2 UV sets was added as the spec requires it.
  • A BRDF look-up table texture is now used.

The Kuesa API has been augmented in a few ways : The Kuesa API now provides a way to pass options to the parser, through the GLTF2Options C++ class and Kuesa.GLTF2Options QML object.

Post-processing filters have seen an overhaul, and can now leverage depth textures. In addition, a few new effects were implemented : depth-of-field and bloom. Bloom can be tested in the car scene demo, and all effects are showcased in the framegraph_scene test project.

Not as shiny as the latest Corvette, but still with a heavy dose of bloom!

Normals and tangents can now be automatically generated when missing (through the help of the MikkTSpace open-source library) – see the GLTF2Options.

gltfEditor (ex-assetpipelineeditor) has gained a few new features, too :

  • It is now possible to configure the exposure, gamma, and the tonemapping algorithm in the settings.
  • The editor is able to save and load camera settings, under the camera pane.
  • Tangents can conveniently be generated from the user interface.

Polly model in the gltfEditor

A few new examples and demos were added :

  • A music box moved by a mechanical arm. Bonus points for who will find the original melody used in the demo 🙂

The music box demo

  • A demo of the various tonemapping algorithms implemented in Kuesa :

The various tonemapping filters available, as showcased in the tonemapping demo

General improvements

The codebase was thoroughly tested on a wide range of hardware ; as such, many fixes in multiple areas of the codebase were implemented. Various HiDPI and multisampling related issues were fixed. For instance, post-processing effects now support multisampling correctly.

Better performance was achieved for the car demo on low-power devices thanks to the use of pregenerated images for the user interface.

The tonemapping and PBR pipelines were also reworked. The changes fix color space issues in the frame graph, in particular with post-processing effects. The PBR rework also fixes various subtle rendering artefacts.

Contributing to Kuesa

An infrastructure change has been put in place during this release cycle : dissatisfied by the pull request UI of Github, we migrated the code review to an open Gerrit instance, linked with our Github repository.

This instance is available at https://kuesa-codereview.kdab.com. Please use it to contribute!

Finally, a big thanks to all the contributors for this new version : Jim Albamont, Robert Brock, Timo Buske, Juan Jose Casafranca, Jean-Michaël Celerier, Wieland Hagen, Sean Harmer, Mike Krus, Paul Lemire, David Morgan, Mauro Persano, Nuno Pinheiro, Allen Winter.

About KDAB

If you like this blog and want to read similar articles, consider subscribing via our RSS feed.

Subscribe to KDAB TV for similar informative short video content.

KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.

The post New features and changes in the Kuesa 1.1.0 release appeared first on KDAB.

Categories: FLOSS Project Planets

About “Qt offering changes 2020”

Planet KDE - Tue, 2020-01-28 04:56

When I read “Qt offering changes 2020” yesterday, my first reaction was to write a pissy blog post. I’m still writing a blog post with my thoughts about the changes, but I’ll be nice. There are three parts to this post: a short recap of my history with Qt and then my thoughts on what this means for KDE, for Krita and for free software.

I started programming using Qt and PyQt when I read about Qt in Linux Journal, which I was subscribing to back in 1996. That means that I’ve been using Qt for about 25 years. I initially wanted to write an application for handling linguistic field data, and I evaluated GTK+, wxWidgets, Qt, Tk, fltk, V and a few others that have been forgotten in the mists of time. I choose Qt because it had great documentation, a consistent API, the most logical (to me…) way of doing things like setting up a window with a menu or handling scrollbars and finally because it made C++ as easy as Java.

I’ve stayed with Qt for 25 years because, through all the vicissitudes, it kept those qualities. Mostly. There are now a lot more modules, most of which aren’t necessary for my work, there are people working on Qt who seem to be a bit ashamed that Qt makes C++ as easy as Java and want to make Qt as computer-sciency as C++, there have been the licensing issues with the QPL, the changes to GPL, to LGPL and then again some modules back to GPL there have been the Nokia years, the Digia times.

But I’ve always felt that I could build on Qt. And the reason for that is the KDE Free Qt Foundation. To summarize: this is a legal agreement that keeps Qt free software. If the Qt company won’t release a version of Qt under a free software license within a year of a release, Qt becomes licensed under the BSD license.

With yesterday’s message, the Qt company is searching the utter boundaries of this agreement. To recap:

  • Long Term Support releases remain commercial only (the post doesn’t mention this, but those releases also need to be released under a free software license within a year to adhere to the agreement, at least to my understanding).
  • Access to pre-built binaries will be restricted: put behind an account wall or be only available to commercial license holders
  • And there’s a new, cheaper license for small companies that they can use to develop, but not deploy their work to customers.

This is a weirdly mixed bag of “changes”. The last one is a bit silly. Even the “commercial” side of Krita is too big to qualify! We’re five people and have a budget of about 125k…

The middle point is worth considering as well. Now there is nothing in any free software license that talks about a duty to make binaries available.

For a very long time, Krita, when part of KOffice, only made source tarballs available. Right now, we, like the Qt company, have binaries for Linux, Windows, macOS and (experimentally) Android. The Windows binaries are for sale in the Windows Store and on Steam, the Linux binaries are for sale on Steam. And all binaries can be downloaded for free from krita.org and other places.

This move by the Qt company would be like the Krita project shutting down the free downloads of our binaries and only make them available in the various stores. It would be legal, but not nice and would cost us hundreds of thousands of users, if not millions. It is hard not to wonder what the cost to the Qt community will be.

The first change, the restriction of the LTS releases to commercial customers has all kinds of unexpected ramifications.

First off, Linux distributions. Disitributions already rarely use LTS releases, and in any case, with Qt 3 and Qt 4 there didn’t use to be any LTS releases. But disitributions do have to keep older versions of Qt around for unported applications for a longer time, so they do need security and bug fixes for those older versions of Qt.

Then there’s the issue of how fixes are going to land in the LTS releases. At the last Qt contributor summit the Qt project decided on a process where all fixes go through “dev” and then are ported to the stable branches/LTS branches. That’s going to break when Qt6 becomes dev: patches won’t apply to Qt 5.

Albert has already blogged about this change as well, but he only really focused on distributions and KDE Plasma; there is of course much more to KDE than the Plasma desktop and Linux distributions.

As for Krita, we’re using Qt 5.12 for our binaries because we carry a lot of patches that would need porting to Qt 5.13 or 5.14 and because Qt 5.13 turned out to be very, very buggy. For Krita, using a stable version of Qt that gets bug fixes is pretty important, and that will be a problem, because we will lose access to those versions.

In my opinion, while we’ve done without stable, LTS releases of Qt for years, it’s inevitable that Qt 5.15 will be forked into a community edition that gets maintained, hopefully not just by KDE people, but by everyone who needs a stable, LGPL licenced release of Qt5 for years to come

Splitting up the Qt community, already responsible for handling a huge amount of code, is not a good idea, but it looks like the Qt company has made it inevitable.

And once there’s a community maintained fork of Qt, would I contribute to the unforked Qt? Probably not. It’s already a lot of work to get patches in, and doing that work twice, nah, not interested. If there’s a maintained community version of Qt 5, would I be interested in porting to Qt 6? Probably not, either. It isn’t like the proposed changes for Qt 6 excite me. And I don’t expect to be the only one.

As for the more intangible consequences of these changes: I’m afraid those aren’t so good. Even in our small Krita community, we’ve had people suggest it might be a good idea to see whether we couldn’t port Krita to, say, Blender’s development platform. This would be a sheer impossible task, but that people start throwing out ideas like that is a clear sign that the Qt company has made Qt much less attractive.

If I were to start a new free software project, would I use Qt? Last Sunday the answer would have been “of course!”. Today it’s “hm, let’s first check alternatives”. If I had a big GTK based project that’s being really hampered by how bad, incomplete and hard to use GTK is, would I consider porting to Qt? Same thing. If the KDE Free Qt Foundation hadn’t that agreement with the Qt company, the answer would probably be no, right now, it’s still probably a yes.

Now as for the actual announcement. I think the way the Qt company represents the changes is actually going to help to harm Qt’s reputation. The announcement is full of weasel-wording…

“General Qt Account requirement” — this means that in order to download Qt binaries, everyone is going to need a Qt account. Apparently this will make open-source users more eager to report bugs, since they will already have an account. And, yay, wonderful, you need an account to access the completely useless Qt marketplace. And it allows, and now we’re getting at the core reason, the Qt company to see which companies are using the open source version of Qt and send salespeople their way. (But only if the people making the accounts are recognizable, of course, not if they make the account with their gmail address.) When I was working for Quby, I was unpleasantly surprised at how expensive Qt is, how little flexibility the Qt company shows when dealing with prospective customers — and how we never downloaded the installer anyway.

“LTS and offline installer to become commercial-only” — this will break every free software project that uses services like travis to make builds that download Qt in the build process. Of course, one can work around that, but the way the Qt company represents this is “We are making this change to encourage open-source users to quickly adopt new versions. This helps maximize the feedback we can get from the community and to emphasize the commercial support available to those with longer product life cycles that rely on a specific Qt version.” Which of course means “our regular releases are actually betas which we expect you freeloaders to test for us, to provide bug fixes for us, which we can use to provide the paying customers with stable releases”.

And yes, theoretically, the main development branch will have all bug fixes, too, and so nobody misses out on those bug fixes, and everyone has stability… Right? The problem is that Qt has become, over the years, bigger and buggier, and I doubt whether releases made fresh off the main development branch will be stable enough to provide, say, a stable version of Krita to our millions of users. Because, apart from all the bug fixes, they will also have all the new regressions.

“Summary”. The Qt Company is committed to the open-source model of providing Qt technology now and in the future and we are investing now more than ever. ” — though only to the extent that the Qt Company is forced to adhere to the open-source model by the KDE Free Qt Foundation.

We believe that these changes are necessary for our business model and the Qt ecosystem as a whole. ” — my fear is that the Qt Company will not survive the fracturing of the Qt ecosystem that this decision practically guarantees.

Categories: FLOSS Project Planets

Guest Post: Current State of ComDaAn: Community Data Analytics, by Christelle Zouein

Planet KDE - Tue, 2020-01-28 03:56

If you remember my previous post, I mentioned that some work has been happening thanks to an intern we had for a few months at enioka Haute Couture. Well, we thought it would be unfair to just take credit of her work and so she wrote a piece about her internship that we’re sharing with the world.

The following is the description of the current state of ComDaAn in her own words. It has been previously published in french on the enioka blog and on her own blog as well

Thanks again Christelle for the wonderful work you did. The stage is yours now!

Current State of ComDaAn: Community Data Analytics, by Christelle Zouein

Faced with the growing number of tools developers use to write and distribute code, we’ve decided to have a closer look at the data footprint these leave behind. ComDaAn analyzes said footprint to offer an insight into the inner workings of developer teams and open sources communities.

It can be used to examine how different communities and teams function by for example looking at the commits of a git repository over time. This can be particularly useful in the case of a technical audit or to select the software “brick” whose community has the best long-term chances.

This article will cover community data analytics in the context of open source development and free software. More specifically it will seek to introduce ComDaAn and explain the newest additions to the project.

A little bit of history

Paul Adams is a developer renowned for his work in the field of free software and his many contributions to the KDE FOSS community. Before retiring from KDE, Adams provided the community with a service in the form of community data visualization using git repositories. To ensure the continuity of the service, Kevin Ottens, Libre software craftsman and developer at enioka Haute Couture, decided to take over.

And so, ComDaAn took form as a way of modernizing Paul Adams’ scripts while staying true to his vision.

That later turned into a complete rewrite with the purpose of creating a solid base for community data analytics in general. The project then became a suite of tools to study and analyze data produced by software communities and teams.

Features

ComDaAn became what it is today thanks to the common efforts of multiple developers. It has many features, most of which will be explained in what follows.

Supported data sources

To conduct analyses on open source communities and software teams, ComDaAn uses the data these entities produce, mainly git repositories, mailing lists and GitLab issues.

Git repositories

Perhaps the most intuitive data source to consider would be git repositories. Indeed, git repositories are a way to store files as well as track the changes that have been made to them. Git is a tool designed to coordinate work between programmers and is currently the one most commonly used by both the free software and proprietary software communities alike.

So, analyzing their git repositories and thus the products they put forth would be the most direct way to study these communities.

This was the starting point for ComDaAn.

Mailing Lists

Mailing lists are a sort of discussion lists or forums where internet users can communicate and discuss certain subjects. Because of the advantages they present over other means of communications, such as the possibility to work offline, that of signing one’s emails via GPG or even using a mail client’s filtering features, they are commonly used in open source communities.

Therefore, analyzing the public mailing lists of an open source community or the private archives of a team of developers, offers an insight into the discussion within them as well as their readiness for discussion with both users and developers. It thus seemed appropriate to add this data source to the project to establish a more complete profile of the community or team studied.

GitLab Issues

Some Git repository managers like GitLab or GitHub offer an issue tracking system (ITS) that manages and maintains lists of issues which consist of a way to measure the work needed to improve a project. Popular amongst developer teams and communities, ITSs have become a tool for software maintenance. Analyzing them therefore allows the scrutiny of this aspect of a project.

Moreover, certain actors of the software community still went unnoticed even with two data sources. And analyzing the issues of a project helps to overcome this problem since it focuses on the lesser known members of the community: those who report bugs (the reporters) and those who discuss them (the commenters).

For these reasons, ComDaAn needed to be expanded to include the analysis and fetching of GitLab issues.

Types of analyses

There are 5 types of analyses in ComDaAn that a user can perform on the supported data sources: activity, size, network, centrality and response time.

In what follows, the team or community mentioned is formed by the authors of the entries and the project by the data source or sources submitted for analysis. The duration of the project thus becomes equal to the duration of all the entries combined. In addition to that, the results obtained represent the active members of the entity in question and not all of the entity.

Activity

The activity analysis can be seen as a tool used to visualize the activity of the members of a community or a team, week by week, for the duration of a certain project. The term “activity” here represents the number of commits, email messages, issues or comments created by the members of the team or community during the period considered.

And so, it becomes possible to identify who has been the most active out of a group. It is also possible to tell when a member has joined or left the team or community considered which gives an idea about the recruitment and retention trends.

The activity visualization consists of a list of the author names and their weekly activity over the life of a project.


Team member activity

Thanks to the above visualization, we can examine the authors that have contributed to the project and the frequency at which they have. And the darker the color, the more active the author has been during the corresponding week.

Size

The size analysis can be seen as a tool that is used to plot the variation of a community’s size and activity over the duration of a project. Unlike the activity analysis which studies the team on a per person basis, the size analysis looks at the team as a whole. Indeed, the analysis produces a scatterplot that, thanks to some smoothing, into two curves, one representing the number of authors in a community as a function of time and one representing the number of entries in the corresponding project as a function of time.


Team activity and size over time

With such a visualization, it becomes possible to reveal certain events that would have otherwise gone unnoticed, by looking at the general tendencies of the team size or activity. In the previous example, an interesting question that might arise would be what event happened during the life of the project that generated an activity peak in 2012.

This visualization is fairly easy to read and consists of a guide to choose more specific analyses on more relevant periods. For instance, it would be worth considering looking at the contributor network (further explained in the following section) of the above project before 2012 and then during 2012 to further investigate the causes of the activity peak.

Network

The network analysis is a tool to show the relationships between the different members of a team or community as well as its structure. To create the graph representing said structure and calculate the centrality of each node or author, we proceed differently for each data source. For git repositories, we mainly consider the changes an author made to a set of files and the more said author has modified files a lot of others have modified, the more they have collaborated with the rest of their team (at the least to synchronize and discuss their modifications). They will appear to be more central to the contributor network. For mailing lists, we mainly use the references in a message. And thus, the more a sender has been referenced by different senders, the larger their weight. And finally, for GitLab issues, we rely on the discussion surrounding the issues of a project which entails that the more an author reports bugs and comments on discussion threads, the larger their weight.


A network of Contributors

It thus becomes easy to identify the key contributors of a certain project or community over a given period. Typically, the more central person or people of a project are those who maintain it. A use case of the network analysis would then be for instance to conduct it on different periods during the life of a project to see if the maintainers have changed. A project that can survive a change of maintainers could be labeled as a more resilient project.

Centrality

Where a network analysis shows the centrality of the different members of a team or a community, a centrality analysis shows the variation of that of an individual author over time. In it, the centrality of an author is calculated similarly to how it is in the network analysis, but instead of considering the duration of the whole project, it considers a smaller interval and calculates the centrality of the author over it. It then repeats the computation over the course of said project. Thus, this type of visualizations displays the variation of the centrality of a member over time. It also displays the activity variation of the author over time and that of the size of the community over the same time.

Instead of studying the entity in question at the community or team scale like its counterparts, the centrality analysis allows us to study each member at the individual scale and their impact on the team or community.


Contributor centrality over time

This particular visualization is the most computationally costly and the hardest to read. Indeed, the amplitude of the centrality plot would vary depending on the size of the team. Therefore, centrality tendency analyses are only relevant over periods of relative stability team/community size-wise (hence the third plot in the above example).

Responsiveness

Essentially, issues are the support that the creators of a certain software offer to its users. So, the study of the response time of a team to reported bugs and issues offers an insight into the maintenance of said software and the dedication of the team to technical support.

This analysis calculates, on one hand, the duration between the creation of the issue and the start of the discussion around it to plot the variation of the response time over time, and on the other the rate of unanswered issues at each point in time.

More specifically, we plot a figure, where the curve representing the variation over time of the response time to issues and a bar chart representing the number of unanswered issues.


Responsiveness to issues over time

We can thus represent in a more tangible way, the dedication of a community or team to the success of their products after deployment. Subsequently such a visualization can for example help in the choice of new tools to adopt.

Interface

The ComDaAn user interface has been designed to be efficient and easy to use. It is a hybrid interface and is made up of a script to collect the data sources from the GitLab API and a python library for parsing, analyzing and displaying data.

The library consists of three functions for the parsing of the three different data sources, of five functions for the five different analyses, and of one function to display the results. It mainly uses “pandas.DataFrame”, making it more universal. It also allows the user to modify these data frames according to their needs as well as to choose the columns they want to use for their analysis. Indeed, to use the library, the user must specify the data set as well as the columns to be considered when calling each analysis function.

To better illustrate, let’s consider the following code snippet:

commits = cd.parse_repositories('~/path_to_repositories') activity = cd.activity(commits, 'id', 'author_email', 'date') cd.display(activity)

Here, we want to perform an activity analysis of some git repositories located at a certain path from the root. We first start by calling the parsing function that corresponds to our data source. That function returns a data frame of the different entries.

Then, we call the activity analysis function on the data frame and specify the columns to be considered.

Finally, we display our result with the display function.

However, to have the authors’ email addresses in our final DataFrame instead of their names, we enter the name of the author email column where the function expects the author names. The result then displays the activity per week for individuals whose emails are those found in the dataset.


Activity analysis displayed by email

Moreover, the display function offers many options. It can receive as a parameter a heterogenous list of elements and displays them differently according to their types. It can also receive a simple pandas data frame and display it. Additionally, to better compare two or more results of the same object type, it is possible to display them on the same figure with the same axes. Examining them would have us look at one figure with different plots instead of repeatedly switch between tabs.

To better illustrate let’s consider this time the following code snippet:

commits = cd.parse_repositories('~/path_to_data') issues = cd.parse_issues('~/path_to_data') commits_ts = cd.teamsize(commits, 'id', 'author_name', 'date') issues_ts = cd.teamsize(issues, 'id', 'author', 'created_at') cd.display([commits_ts, issues_ts])

Here, we decided to use data from different sources but apply the same analysis to them and then display them at once. The two different figures are then overlaid on the same figure.


Team activity and size using both team commits and issues

Finally, thanks to the separation of the different stages of the ComDaAn process, it is possible to avoid potential redundancies. Indeed, the parsing which is the most time-consuming step of the whole process is done once per data source where it used to be done by each script called and the display is optimized. The global execution time is thus smaller.

And now …

During my four-month internship at enioka Haute Couture, I worked on ComDaAn And more specifically on optimizing the code performance wise, on adding mailing lists and issues as additional data sources, on using the LOWESS regression for smoothing of the different curves displayed, and finally on the design and programming of the interface.

The ComDaAn project has evolved over time thanks to the work and dedication of many contributors. In the same manner, it will continue to evolve and find new ways to serve community data analytics as well as the open source community.

Christelle Zouein

Categories: FLOSS Project Planets

Steve Kemp: Initial server migration complete..

Planet Debian - Tue, 2020-01-28 02:30

So recently I talked about how I was moving my email to a paid GSuite account, that process has now completed.

To recap I've been paying approximately €65/month for a dedicated host from Hetzner:

  • 2 x 2Tb drives.
  • 32Gb RAM.
  • 8-core CPU.

To be honest the server itself has been fine, but the invoice is a little horrific regardless:

  • SB31 - €26.05
  • Additional subnet /27 - €26.89

I'm actually paying more for the IP addresses than for the server! Anyway I was running a bunch of virtual machines on this host:

  • mail
    • Exim4 + Dovecot + SSH
    • I'd SSH to this host, daily, to read mail with my console-based mail-client, etc.
  • www
    • Hosted websites.
    • Each different host would run an instance of lighttpd, serving on localhost:XXX running under a dedicated UID.
    • Then Apache would proxy to the right one, and handle SSL.
  • master
    • Puppet server, and VPN-host.
  • git
  • ..
    • Bunch more servers, nine total.

My plan is to basically cut down and kill 99% of these servers, and now I've made the initial pass:

I've now bought three virtual machines, and juggled stuff around upon them. I now have:

  • debian - €3.00/month
  • dns - €3.00/month
    • This hosts my commercial DNS thing
    • Admin overhead is essentially zero.
    • Profit is essentially non-zero :)
  • shell - €6.00/month
    • The few dynamic sites I maintain were moved here, all running as www-data behind Apache. Meh.
    • This is where I run cron-jobs to invoke rss2email, my google mail filtering hack.
    • This is also a VPN-provider, providing a secure link to my home desktop, and the other servers.

The end result is that my hosting bill has gone down from being around €50/month to about €20/month (€6/month for gsuite hosting), and I have far fewer hosts to maintain, update, manage, and otherwise care about.

Since I'm all cloudy-now I have backups via the provider, as well as those maintained by rsync.net. I'll need to rebuild the shell host over the next few weeks as I mostly shuffled stuff around in-place in an adhoc fashion, but the two other boxes were deployed entirely via Ansible, and Deployr. I made the decision early on that these hosts should be trivial to relocate and they have been!

All static-sites such as my blog, my vanity site and similar have been moved to netlify. I lose the ability to view access-logs, but I'd already removed analytics because I just don't care,. I've also lost the ability to have custom 404-pages, etc. But the fact that I don't have to maintain a host just to serve static pages is great. I was considering using AWS to host these sites (i.e. S3) but chose against it in the end as it is a bit complex if you want to use cloudfront/cloudflare to avoid bandwidth-based billing surprises.

I dropped MX records from a bunch of domains, so now I only receive email at steve.fi, steve.org.uk, and to a lesser extent dns-api.com. That goes to Google. Migrating to GSuite was pretty painless although there was a surprise: I figured I'd setup a single user, then use aliases to handle the mail such that:

  • debian@example -> steve
  • facebook@example -> steve
  • webmaster@example -> steve

All told I have about 90 distinct local-parts configured in my old Exim setup. Turns out that Gsuite has a limit of like 20 aliases per-user. Happily you can achieve the same effect with address maps. If you add an address map you can have about 4000 distinct local-parts, and reject anything else. (I can't think of anything worse than having wildcard handling; I've been hit by too many bounce-attacks in the past!)

Oh, and I guess for completeness I should say I also have a single off-site box hosted by Scaleway for €5/month. This runs monitoring via overseer and notification via purppura. Monitoring includes testing that websites are up, that responses contain a specific piece of text, DNS records resolve to expected values, SSL certificates haven't expired, & etc.

Monitoring is worth paying for. I'd be tempted to charge people to use it, but I suspect nobody would pay. It's a cute setup and very flexible and reliable. I've been pondering adding a scripting language to the notification - since at the moment it alerts me via Pushover, Email, and SMS-messages. Perhaps I should just settle on one! Having a scripting language would allow me to use different mechanisms for different services, and severities.

Then again maybe I should just pay for pingdom, or similar? I have about 250 tests which run every two minutes. That usually exceeds most services free/cheap offerings..

Categories: FLOSS Project Planets

PyCharm: PyCharm 2020.1 EAP starts now

Planet Python - Tue, 2020-01-28 00:57

There are two types of people in the world: those who can wait to open a package they’ve received, and people like me, who need to see what’s inside this very second.

PyCharm isn’t delivered in the mail though, and that’s why we have something even better for impatient people. The early access program (EAP) shows you what’s in the package a couple months before you get it. Take a sneak peek, and get PyCharm’s first EAP now!

New in PyCharm JetBrains Mono

We recently announced our font designed especially for programming. This means that we’ve optimized the font for reading vertically, and other optimizations like code-specific ligatures. If you haven’t seen it yet, be sure to check out the page where we introduce JetBrains Mono, it clearly shows how it makes programming a more enjoyable experience.

From PyCharm 2020.1 onward, this font is chosen as a default in the editor. If you had previously configured another font, be sure to try out JetBrains Mono to see if you like it. You can configure the font in Settings | Editor | Font.

Further Improvements
  • Some Version Control features just got better! On the Branches popup (Find action, Ctrl/Cmd-Shift-A, and then look for ‘Branches’), there’s now a refresh icon that will fetch changes from the git remote and a search box that allows you to quickly find your branch.
  • We’ve updated the ‘Python Debug Server’ run configuration. It used to be called ‘Python Remote Debug’, and we think this name makes clearer what it does. When you run this configuration, PyCharm will listen to an incoming connection from your script. To use it, you need to modify your script, and ensure that the parameters to the settrace call are correct: it should be the hostname how the script (from where it’s running) can access the host where the IDE is running, and the appropriate port. You need to ensure yourself that the hostname correctly resolves for the script, and that all firewalls are appropriately configured for the connection to succeed. A common way to punch through firewalls is to use this feature together with SSH remote forwarding. If you want PyCharm to do this for you, a remote interpreter can handle all the details in the background.
  • To see everything that’s new in this release, check out the release notes.
Interested?

Download this EAP from our website. Alternatively, you can use the JetBrains Toolbox App to stay up to date throughout the entire EAP.

If you’re on Ubuntu 16.04 or later, you can use snap to get PyCharm EAP, and stay up to date. You can find the installation instructions on our website.

Categories: FLOSS Project Planets

The Qt Company is stopping Qt LTS releases. We (KDE) are going to be fine :)

Planet KDE - Mon, 2020-01-27 18:02

Obvious disclaimer, this is my opinion, not KDE's, not my employer's, not my parents', only mine ;)

Big news today is that Qt Long-term-supported (LTS) releases and the offline installer will become available to commercial licensees only.

Ignoring upcoming switch to Qt6 scenario for now, how bad is that for us?

Let's look at some numbers from our friends at repology.

At this point we have 2 Qt LTS going on, Qt 5.9 (5.9.9 since December) and Qt 5.12 (5.12.6 since November).

How many distros ship Qt 5.9.9? 0. (there's macports and slackbuilds but none of those seem to provide Plasma packages, so I'm ignoring them)

How many distros ship Qt 5.12.6? 5, Adélie Linux, Fedora 30, Mageia 7, OpenSuse Leap 15.2, PCLinux OS (ALT Linux and GNU Guix also do but they don't seem to ship Plasma). Those are some bigger names (I'd say specially Fedora and OpenSuse).

On the other hand Fedora 28 and 29 ship some 5.12.x version but have not updated to 5.12.6, Opensuse Leap 15.1 has a similar issue, it's stuck on 5.9.7 and did not update to 5.9.9 and so is Mageia 6 which is stuck on Qt 5.9.4

Ubuntu 19.04, 19.08 and 20.04 all ship some version of Qt 5.12 (LTS) but not the lastest version.

On the other a few of other "big" distros don't ship Qt LTS, Arch and Gentoo ship 5.14, our not-distro-distro Neon is on 5.13 and so is flatpak.

As I see it, the numbers say that while it's true that some distros are shipping the latest LTS release, it's not all of them by far, and it looks more like an opportunistic use, the LTS branch is followed for a while in the last release of the distro, but the previous ones get abandoned at some point, so the LTS doesn't really seem to be used to its fully potential.

What would happen if there was no Qt LTS?

Hard to say, but I think some of the "newer" distros would actually be shipping Qt 5.13 or 5.14, and in my book that's a good thing, moving users forward is always good.

The "already released" distros is different story, since they would obviously not be updating from Qt 5.9 to 5.14, but as we've seen it seems that most of the times they don't really follow the Qt LTS releases to its full extent either.

So all in all, I'm going to say not having Qt LTS releases is not that bad for KDE, we've lived for that for a long time (remember there has only been 4 Qt LTS, 4.8, 5.6, 5.9 and 5.12) so we'll do mostly fine.

But What about Qt 5.15 and Qt 6 you ask!
Yes, this may actually be a problem, if all goes to plan Qt 5.15 will be released in May and Qt 6.0 in November, that means we will likely get up to Qt 5.15.2 or 5.15.3 and then that's it, we're moving to Qt 6.0

Obviously KDE will have to move to Qt 6 at some point, but that's going to take a while (as example Plasma 5 was released when Qt was at 5.3) so for let's say that for a year or two we will still be using Qt 5.15 without any bugfix releases.

That can be OK if Qt 5.15 ends being a good release or a problem if it's a bit buggy. If it's buggy, well then we'll have to figure out what to do, and it'll probably involve some kind of fork somewhere, be it by KDE (qt already had that for a while in ancient history with qt-copy) or by some other trusted source, but let's hope it doesn't get to that, since it would mean that there's two set of people fixing bugs in Qt 5.15, The Qt Company engineers and the rest of the world, and doing the same work twice is not smart.

Categories: FLOSS Project Planets

Norbert Preining: SUSI.AI release 20200120: Desktop and Smart Speaker

Planet Debian - Mon, 2020-01-27 17:31

More than a month has passed, but the winter holidays allowed me to update, fix, and stream line a lot of corners in SUSI.AI. And above all, work on a desktop version that can easily be installed. Thus, the FOSSASIA Team finally can release a SUSI.AI 2020-01-20 of SUSI.AI, the privacy aware personal assistant.

Since long it has been possible to install and run SUSI.AI on Debian based desktops, and recent months have brought (partial) support for other distributions (RedHat, SUSE etc). This is the first release that provides an installation package for desktop environments that allow for easy installation. As usual, we also ship an image for the SUSI.AI Smart Speaker based on Raspberry Pi.

Changes in this release are:

  • Much improved documentation concerning necessary requirements
  • Separate installers for required dependencies and actual SUSI.AI, with support for different packaging infrastructure
  • fixed problems in portaudio due to upgrade to Debian/buster
  • reworked hotword/audio system for increased stability
  • release for desktop environments: fully relocatable SUSI.AI folder
  • susi-config program now can install .desktop files, systemd service files, and link binaries to directories in the PATH
  • initial work towards DeepSpeech support
  • many fixes and internal improvements

We are looking forward to feedback and suggestions, improvements, pull request! Please send either issues to one of the git repositories, or join us at one of the gitter channels (susi_hardware, susi_server, susi_android, susi_webclient, or generally at fossasia)!

Categories: FLOSS Project Planets

FSF Blogs: GNU Spotlight with Mike Gerwitz: 16 new GNU releases in January!

GNU Planet! - Mon, 2020-01-27 16:42

For announcements of most new GNU releases, subscribe to the info-gnu mailing list: https://lists.gnu.org/mailman/listinfo/info-gnu.

To download: nearly all GNU software is available from https://ftp.gnu.org/gnu/, or preferably one of its mirrors from https://www.gnu.org/prep/ftp.html. You can use the URL https://ftpmirror.gnu.org/ to be automatically redirected to a (hopefully) nearby and up-to-date mirror.

A number of GNU packages, as well as the GNU operating system as a whole, are looking for maintainers and other assistance: please see https://www.gnu.org/server/takeaction.html#unmaint if you'd like to help. The general page on how to help GNU is at https://www.gnu.org/help/help.html.

If you have a working or partly working program that you'd like to offer to the GNU project as a GNU package, see https://www.gnu.org/help/evaluation.html.

As always, please feel free to write to us at maintainers@gnu.org with any GNUish questions or suggestions for future installments.

Categories: FLOSS Project Planets

GNU Spotlight with Mike Gerwitz: 16 new GNU releases in January!

FSF Blogs - Mon, 2020-01-27 16:42

For announcements of most new GNU releases, subscribe to the info-gnu mailing list: https://lists.gnu.org/mailman/listinfo/info-gnu.

To download: nearly all GNU software is available from https://ftp.gnu.org/gnu/, or preferably one of its mirrors from https://www.gnu.org/prep/ftp.html. You can use the URL https://ftpmirror.gnu.org/ to be automatically redirected to a (hopefully) nearby and up-to-date mirror.

A number of GNU packages, as well as the GNU operating system as a whole, are looking for maintainers and other assistance: please see https://www.gnu.org/server/takeaction.html#unmaint if you'd like to help. The general page on how to help GNU is at https://www.gnu.org/help/help.html.

If you have a working or partly working program that you'd like to offer to the GNU project as a GNU package, see https://www.gnu.org/help/evaluation.html.

As always, please feel free to write to us at maintainers@gnu.org with any GNUish questions or suggestions for future installments.

Categories: FLOSS Project Planets

FSF Blogs: LibrePlanet 2020: We'll see you at the Back Bay Events Center in Boston, MA!

GNU Planet! - Mon, 2020-01-27 12:55

We at the Free Software Foundation (FSF) are excited to say that the Dorothy Quincy suite of Boston's very own Back Bay Events Center will be the home of this year's LibrePlanet conference! We've taken the grand tour and couldn't be happier about our choice of location. We're confident that the Events Center will be a great host for the technology and social justice conference we've all come to know and love. It's just the right place for us (and the movement) to take our next steps in freeing the future.

The Events Center is providing LibrePlanet with its own entrance and a dedicated and speedy Internet connection for the livestream, and is close to both public transportation and the FSF headquarters itself. As in past years, we'll have ample space for an exhibit hall and free software workshops, as well as the ever popular "hallway track," where you can engage with other attendees in conversations on contributing to free software projects.

On the Events Center Web site, you will find accommodation and transportation suggestions that will pair nicely with those we've put up on the LibrePlanet 2020 site. The Back Bay Events Center is located at the corner of Berkeley and Stuart Street, and is close by the Back Bay stop of the Orange Line MBTA train and the Arlington stop of the Green Line MBTA train.

If you have attended LibrePlanet in past years but are generally unfamiliar with the Boston area, please note that LibrePlanet 2020 will be held in Boston and not the nearby city of Cambridge.

As in past years, you can expect the venue to be fully accessible, and equipped with enough network horsepower to drive the conference livestream. For more information on the venue's perks and services, visit the Back Bay Events Center Web site, or reach out to campaigns@fsf.org. And if you have yet to register for the LibrePlanet 2020 conference, now is the time to do so!

LibrePlanet depends on the community for its success. One way you can help us is by donating to help sponsor an attendee to come to LibrePlanet, and assist us in making the conference a truly global one. If you're interested in volunteering, please write us at resources@fsf.org.

All of us here at the FSF are deep in the planning process, but we couldn't be more excited about seeing you in person -- especially if it is your first time. (No worries, it's my first LibrePlanet conference as well!) Let's use the time we have together to the fullest, and make LibrePlanet 2020 go down in history as the place where we made great strides to "Free the Future."

Categories: FLOSS Project Planets

Pages