FLOSS Project Planets

Glyph Lefkowitz: Inbox Zero, Cost: Zero

Planet Python - Mon, 2022-05-02 19:27

One consistent bit of feedback that I’ve received on my earlier writing about email workflow is that I didn’t include a concrete enough set of instructions for getting started with task-management workflow, particularly with low-friction options that are available for people who don’t necessarily have $100 per year to drop on the cadillac of task-management applications.

Given that the piece seems to be enjoying a small resurgence of attention, I’ve significantly expanded the “Make A Place For Tasks” section of that article, with:

  • more no-cost, low-friction options for getting started (if you’re stuck on this step “if you use Gmail, just start using Google Tasks” is the main takeaway)
  • a guide for how to evaluate a task-management application for yourself, if you are trying to pick something that fits your work style better
  • several links to the specific “create a task from an email” tools and workflows for each app

It was nice to be doing this update now, because in the years since that piece was published, almost every major email application has added task-management features, or upgraded them into practical usability; gone are the times when properly filing your emails into clearly-described tasks was an esoteric feature that you needed expensive custom software for.

Categories: FLOSS Project Planets


Planet KDE - Mon, 2022-05-02 18:00

It was all whirlwind and heat. One weekend we hit the road and went to Italy. Where there was the Linux App Summit (LAS) 2022. I mentioned I was going, and then I went! And it was all I needed it to be.

Linux App Summit banner

I chose Ruth and Heather to listen to, and posted some bits on the socials as the conference progressed. I acted as MC for the main auditorium, so that was two days of introducing speakers, announcing coffee breaks, and dealing with the unexpected. Oh, and running microphones for questions. I would like to thank Dr. Britt Yazel of the Ubuntu folk for being an absolute stellar room-runner – dealing with the technical things and keeping track of time. Also for always having a question if the room was a bit quiet.

Anisa, Caroline and Kristi did a great job wrangling people, places and things. Dani was there for all the things that needed doing, Giulio for the desk and Heather not only gave a talk but also climbed ladders. Regina chased a bunch of social and future-PR items, and Kenny watched over streaming. Truly, a conference like this rests on the shoulders of two dozen people working their butts off for months beforehand, and then being largely invisible during the conference itself.

The people from iiriti were really cool on the A/V technical front – I don’t think I’ve previously dealt with such a slick setup and helpful people.

I’ve probably not mentioned another dozen people; you were all great.

It turned out the local gaming group in Rovereto had their regular get-together at the bar during the conference, so at one point I was playing “Project L” with some random passer-by and Patrick from openSUSE – these are the best parts of conferences.

Linux App Summit stage

Here’s a shout-out to Codethink and Canonical, two sponsors of the conference who provided food (the cake was not a lie, it was Jellyfish, and the coffee-and-brioches were necessary for life itself).

The friendly folks from openSUSE – always a pleasure to talk with Doug – brought a Geeko to liven things up, and MBition, while not very visible at the conference, provided financial support for the conference. Slimbook and Tuxedo supported the conference as well.

You can see all my favorites in that picture of the stage, Slimbook with openSUSE and FreeBSD and cat stickers and KDE.

Tip of the hat to Neil and Rob from GNOME, and all the speakers. I’ll see you next year!

Categories: FLOSS Project Planets

DevCollaborative - Planet Drupal: Maxlength 2.0.0-RC2 - A More Accurate and Easy to Configure Character Limit

Planet Drupal - Mon, 2022-05-02 17:05
The latest version of Maxlength fixes a user permission, character count discrepancies and more. Plus an improved the UI based on user feedback.
Categories: FLOSS Project Planets

Chapter Three: Avoid a Drupal 9 Migration Headache: Use NEXT-DRUPAL to Upgrade Your Drupal 7 Site

Planet Drupal - Mon, 2022-05-02 15:35
Drupal 7 and PHP’s end-of-life are right around the corner and many people are thinking about the best path forward.* The three options tend to be: a.) traditional “Lift and Shift” to Drupal 9, b.) “Lift, Shift and Improve a few things,” and c.) “totally rethink,” and rebuild the site. Each option produces a complex Drupal 9 site. Many Drupal 7 sites are layers of back-end code, complex front-end code, and content types that have evolved over the years. At first glance, the upgrade process seems insurmountably expensive, and not worth it.  But we think there’s a better way. Chapter Three believes our Drupal 9 and Next.js integration, NEXT-Drupal (www.Next-Drupal.org) is a modern solution to upgrade your Drupal 7 site, giving you excellent installation of Drupal 9 and a great decoupled front-end! In other words, a true headless CMS. The best of both worlds!
Categories: FLOSS Project Planets

Talking Drupal: Talking Drupal #345 - Live from DrupalCon

Planet Drupal - Mon, 2022-05-02 14:00

Today we are talking about DrupalCon 2022.


  • First in person DrupalCon
  • Most impactful thoughts from DriesNote
  • Best Sessions
  • Next year in Drupal
  • New technologies
  • What are you going to try on the flight home
  • How was your session?
  • Any interesting people or conversations?
  • Swag
Resources Guests

AmyJune Hineline - @volkswagenchick Mike Miles - @mikemiles86 Stephen Cross - stephencross.com - @stephencross


John Picozzi - www.epam.com @johnpicozzi Chris Wells - redfinsolutions.com - @chrisfromredfin

Categories: FLOSS Project Planets

Ben's SEO Blog: What Is Drupal?

Planet Drupal - Mon, 2022-05-02 12:12
What Is Drupal? Get some key concepts about Drupal as well as some useful tips before jumping into SEO on a Drupal website for your company and clients. Tracy Cooper Mon, 05/02/2022 - 11:12
Categories: FLOSS Project Planets

DevCollaborative - Planet Drupal: Affordable, Quality Hosting Alternatives: How to Set Up Drupal on Amezmo

Planet Drupal - Mon, 2022-05-02 11:05
Hosting services with built-in tools based on modern development best practices save dev teams time and headaches, and save our clients money. Although Pantheon is a gold standard in modern hosting in terms of quality, it’s not a good fit for every organization we serve in terms of its pricing model. So we have spent the last few years actively searching for affordable, high quality alternatives that still meet our requirements and standards. Our most successful experiment to date has been on Amezmo hosting. 
Categories: FLOSS Project Planets

Python Morsels: Unicode character encodings

Planet Python - Mon, 2022-05-02 11:00

When working with text files in Python, it's considered a best practice to specify the character encoding that you're working with.

Table of contents

  1. All input starts as raw bytes
  2. Encoding strings into bytes
  3. Decoding bytes into strings
  4. Specifying a character encoding when opening files
  5. Be careful with your character encodings
  6. Summary

All input starts as raw bytes

When you open a file in Python, the default mode is r or rt, for read text mode:

>>> with open("my_file.txt") as f: ... contents = f.read() ... >>> f.mode 'r' >>> with open("my_file.txt") as f: ... contents = f.read() ... >>> f.mode 'r'

Meaning when we read our file, we'll get back strings that represent text:

>>> contents 'This is a file ✨\n' >>> contents 'This is a file ✨\n'

But that's not what Python actually reads from disk.

If we open a file with the mode rb and read from our file we'll see what Python sees; that is bytes:

>>> with open("my_file.txt", mode="rb") as f: ... contents = f.read() ... >>> contents b'This is a file \xe2\x9c\xa8\n' >>> type(contents) <class 'bytes'> >>> with open("my_file.txt", mode="rb") as f: ... contents = f.read() ... >>> contents b'This is a file \xe2\x9c\xa8\n' >>> type(contents) <class 'bytes'>

Bytes are what Python decodes to make strings.

Encoding strings into bytes

If you have a string …

Read the full article: https://www.pythonmorsels.com/unicode-character-encodings-in-python/
Categories: FLOSS Project Planets

Real Python: Python's min() and max(): Find Smallest and Largest Values

Planet Python - Mon, 2022-05-02 10:00

Python’s built-in min() and max() functions come in handy when you need to find the smallest and largest values in an iterable or in a series of regular arguments. Even though these might seem like fairly basic computations, they turn out to have many interesting use cases in real-world programing. You’ll try out some of those use cases here.

In this tutorial, you’ll learn how to:

  • Use Python’s min() and max() to find smallest and largest values in your data
  • Call min() and max() with a single iterable or with any number of regular arguments
  • Use min() and max() with strings and dictionaries
  • Tweak the behavior of min() and max() with the key and default arguments
  • Use comprehensions and generator expressions as arguments to min() and max()

Once you have this knowledge under your belt, then you’ll be prepared to write a bunch of practical examples that will showcase the usefulness of min() and max(). Finally, you’ll code your own versions of min() and max() in pure Python, which can help you understand how these functions work internally.

Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you’ll need to take your Python skills to the next level.

To get the most out of this tutorial, you should have some previous knowledge of Python programming, including topics like for loops, functions, list comprehensions, and generator expressions.

Getting Started With Python’s min() and max() Functions

Python includes several built-in functions that make your life more pleasant and productive because they mean you don’t need to reinvent the wheel. Two examples of these functions are min() and max(). They mostly apply to iterables, but you can use them with multiple regular arguments as well. What’s their job? They take care of finding the smallest and largest values in their input data.

Whether you’re using Python’s min() or max(), you can use the function to achieve two slightly different behaviors. The standard behavior for each is to return the minimum or maximum value through straightforward comparison of the input data as it stands. The alternative behavior is to use a single-argument function to modify the comparison criteria before finding the smallest and largest values.

To explore the standard behavior of min() and max(), you can start by calling each function with either a single iterable as an argument or with two or more regular arguments. That’s what you’ll do right away.

Calling min() and max() With a Single Iterable Argument

The built-in min() and max() have two different signatures that allow you to call them either with an iterable as their first argument or with two or more regular arguments. The signature that accepts a single iterable argument looks something like this:

min(iterable, *[, default, key]) -> minimum_value max(iterable, *[, default, key]) -> maximum_value

Both functions take a required argument called iterable and return the minimum and maximum values respectively. They also take two optional keyword-only arguments: default and key.

Note: In the above signatures, the asterisk (*) means that the following arguments are keyword-only arguments, while the square brackets ([]) denote that the enclosed content is optional.

Here’s a summary of what the arguments to min() and max() do:

Argument Description Required iterable Takes an iterable object, like a list, tuple, dictionary, or string Yes default Holds a value to return if the input iterable is empty No key Accepts a single-argument function to customize the comparison criteria No

Later in this tutorial, you’ll learn more about the optional default and key arguments. For now, just focus on the iterable argument, which is a required argument that leverages the standard behavior of min() and max() in Python:

>>>>>> min([3, 5, 9, 1, -5]) -5 >>> min([]) Traceback (most recent call last): ... ValueError: min() arg is an empty sequence >>> max([3, 5, 9, 1, -5]) 9 >>> max([]) Traceback (most recent call last): ... ValueError: max() arg is an empty sequence

In these examples, you call min() and max() with a list of integer numbers and then with an empty list. The first call to min() returns the smallest number in the input list, -5. In contrast, the first call to max() returns the largest number in the list, or 9. If you pass an empty iterator to min() or max(), then you get a ValueError because there’s nothing to do on an empty iterable.

An important detail to note about min() and max() is that all the values in the input iterable must be comparable. Otherwise, you get an error. For example, numeric values work okay:

>>>>>> min([3, 5.0, 9, 1.0, -5]) -5 >>> max([3, 5.0, 9, 1.0, -5]) 9

These examples combine int and float numbers in the calls to min() and max(). You get the expected result in both cases because these data types are comparable.

However, what would happen if you mixed strings and numbers? Check out the following examples:

>>>>>> min([3, "5.0", 9, 1.0, "-5"]) Traceback (most recent call last): ... TypeError: '<' not supported between instances of 'str' and 'int' >>> max([3, "5.0", 9, 1.0, "-5"]) Traceback (most recent call last): ... TypeError: '>' not supported between instances of 'str' and 'int' Read the full article at https://realpython.com/python-min-and-max/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Codementor: Functions in Python

Planet Python - Mon, 2022-05-02 09:59
In This Article We are going to learn how functions will work. Let's first discuss What exactly is a Python Function. In Python, Function is a group of related statements that performs a specific...
Categories: FLOSS Project Planets

Kdenlive 22.04 released

Planet KDE - Mon, 2022-05-02 09:44
The Kdenlive team is happy to announce the release of version 22.04. This development cycle comes with more than 300 commits, mostly focused on stability and polishing, ranging from packaging all the way up to user interface enhancements.

Besides the improvements to the Windows and macOS versions, Kdenlive is now runs on Apple’s M1 architecture and includes initial support for full range 10-bit color on all platforms — although note that 10-bit color does not work with effects yet. Kdenlive also automatically offers to transcode variable frame-rate videos to an editing-friendly format, and some filters, like Blur, Lift/Gama/Gain, Vignette and Mirror, are now slice threaded, which improves rendering speeds.

Encouraging support among users is one of our priorities and that’s why Kdenlive introduces Effect Templates in version 22.04. Effect Templates are custom effects that can be shared with other community members through the KDE Store and can be downloaded directly into Kdenlive. The store is already open and you can contribute your effects too!

The speech recognition interface got improvements to the highlight color of selected text, font size and it has been appropriately renamed to Speech Editor. Other changes include HighDPI and low resolution screen support, improved OpenTimelineIO handling, ASS subtitle fixes and added CR2, ARW and JP2 image formats.

The render dialog received an interface rewrite, drastically improving usability while giving the user more power by adding a new custom profile creation interface.

Improved render window



New preset creation window


Another highlight is the ability to render multiple videos by zones using the timeline guides.




The Icon View mode in the Project Bin received a major facelift as well.


Full log
  • Ensure audio target track is remembered if we click on a bin clip with the same number of audio streams. Commit. See bug #452495
  • Fix delete render preset on windows not working. Commit.
  • Remove text label. Commit.
  • Fix show clip properties action always disabled. Commit.
  • Fix tests. Commit.
  • Embed time remap ui in a scrollview so that Kdenlive can be used on small resolution (1024×768). Commit.
  • Fix speech editor using large horizontal spacing when selected bin clip has long name. Commit.
  • Fix timeline clip selection broken after a move undo operation. Commit.
  • Enable highdpi. Commit.
  • Mac OS: fix access to microphone. Commit.
  • Fix monitor image size with non integer screen scaling. Commit.
  • Fix app focus lost on Windows when exiting monitor fullscreen. Commit.
  • Switch from QQuickView to QQuickWidget – fixes broken playback on Mac OS. Commit.
  • Fix several cases of timeline losing focus. Commit.
  • Correctly update “apply” button on monitor change. Commit.
  • Make monitor detection more robust for fullscreen mode. Commit.
  • Fix resetting effect does not clear timeline keyframe view, resulting in possible crash. Commit.
  • Don’t propose rtaudio backend if not available. Commit.
  • Fix layout warning. Commit.
  • Fix play zone seeking to first frame of timeline. Commit.
  • Fix import keyframes importing outside clip out. Commit.
  • Remove confusing “autorotate” checkbox in transcode to edit friendly. Commit.
  • Code quality fixes. Commit.
  • Fix fullscreen monitor selection doesn’t work on Windows. Commit.
  • Fix possible crash on exit. Commit.
  • Don’t query producer length on each frame. Commit.
  • Fix speed not saved in custom render profiles. Commit.
  • Code quality fixes. Commit.
  • Ensure we use the breeze widget style on first run in Mac. Commit.
  • Make progress bar for effects more visible (for ex. in motion tracker). Commit.
  • Fix project duration not updated on bin clip deletion. Commit.
  • Fix timeline focus issues on drag & drop, fix mouse position and project duration in timeline toolbar not consistently updated. Commit.
  • Fix no speech end time in analysed speech. Commit.
  • Ignore audio files album art. Commit.
  • Fix typo (missing space). Commit.
  • Fix last silence analysis in speech to text, small drawing fixes. Commit.
  • Fix creating guides from project notes. Commit.
  • Fix line feed lost on project notes paste. Commit.
  • Add invert param to luma mix to allow reversing direction of transition. Commit.
  • Only save bin thumbnail on project save to avoid displaying incorrect thumb after unsaved project change. Commit.
  • Fix freeze on add clip recently introduced. Commit.
  • Fix thumbnail cache bug causing incorrect thumbs to sometimes display after saving project. Commit.
  • Speech to text: cleaner html output for project files, fix work selection not really working. Commit.
  • Fix foxus issue on effect drop. Commit.
  • Smaller drag image in bin, also show it in icon view mode. Commit.
  • Fix startup warning. Commit.
  • Fix timeline focus issues on drag and drop. Commit.
  • Fix timeline scrolling below zero and timeline drag/drop bug. Commit.
  • [Renderer] Fix wrongly inverted logic to hide “Generate Script” button. Commit.
  • Fixes for saving and editing render presets. Commit.
  • [Render Presets] GOP and B-Frames params: enable only if it makes sense. Commit.
  • Fix possible crash in bin when selecting a clip. Commit.
  • [Renderer] Reset DAR to 1:1 on resolution override to avoid errors. Commit.
  • [Render Presets] Combo Box for PAR to prevent render errors. Commit.
  • Fix rubberband selection on scroll, and don’t overlap track headers, fix move clip + mouse wheel. Commit. Fixes bug #417209
  • Fix razor tool misbehaviour on subtitle track. Commit.
  • Fix keymap info on subtitle track. Commit.
  • Fix timecode rounding for fps like 23.98. Commit. Fixes bug #435213
  • Fix double warning and duplicate folder on manage cache data deletion. Commit. Fixes bug #434754
  • Keep focus on clip monitor after editing marker. Commit. Fixes bug #433595
  • Right click on a timeline guide seeks to its position and shows context menu. Commit. Fixes bug #441014
  • Fix editing title/color clips does not invalidate timeline preview. Commit. Fixes bug #437427
  • Titler: remember and restore last used text alignment. Commit. See bug #413572
  • Bin: tree view: hide audio/video icons for audio or video only clips. icon view: hide audio/video icons depending on zoom level, fix tag color covering thumb on drop. Commit.
  • [Render Widget] Prettify by removing some frames. Commit.
  • [Render Presets] Always disable “Scanning” properly if “Progressive”. Commit.
  • [Renderer] Fix preset gets deleted if saving is canceled. Commit.
  • Fix selection in bin icon view. Commit.
  • Fix previous commit (inverted logic). Commit.
  • Fix extract frame using proxy clips. Commit.
  • Fix “Clip” menu not properly update. Commit.
  • Fix regression “Extract Audio” always hidden. Commit.
  • [Render Widget] Fix threads param not updated. Commit.
  • Make audio/video usage icons more visible. Commit.
  • Spot remover, start with a small zone, not full screen. Commit.
  • Fix crash and corruption (disappearing effect) when dropping a clip close to 0 in timeline. Commit.
  • Fix open clip in bin when multiple bins and in icon view. Commit.
  • Multiple fixes for bin. Commit.
  • Improve tooltip on mix enter/exit/resize, display number of selected clips and duration in status bar. Commit.
  • Rename “Text edit” widget to more descriptive “Speech editor”. Commit.
  • [Renderer] Fix: quality slider was reversed. Commit.
  • Fix minor typos. Commit.
  • [Render Presets] Fix: “channels” has a UI control. Commit.
  • [Render] Don’t allow to set rate control for pcm_ audio codecs. Commit.
  • Fix crash and corruption on mix resize, add tests. Commit.
  • Turn off seek on add effect on first start. Commit.
  • Fix reset tracker effect incorrectly moves analyse rectangle. Commit.
  • Proper approach to set default render preset. Commit.
  • Fix project storage folder incorrectly stored, resulting on modified project on open. Commit.
  • [Render Widget] Refactoring and feature extension. Commit. Fixes bug #415610
  • Fix clip task sometimes not performed because of older pending task. Commit.
  • Merge disable autorotate and force rotation on same line in clip properties, fix disable video stream. Commit.
  • Fix transcoding to correctly adjust to audio/video only transcode when required, adjust button text to inform user. Commit.
  • Fix effect stack cleared on clip copy. Commit.
  • Fix defects. Commit.
  • Keyframe fixes: fix keyframe view not updated on clip resize and timeline position in keyframe view not updated on clip move. Commit.
  • Fix timeline preview invalidated by composition on project load. Commit.
  • Add option to seek to clip on add effect (enabled by default, can be disabled from Kdenlive Settings > timeline). Commit.
  • Slideshow clips: display image duration and count in clip properties. Commit.
  • Slideshow animations now use external config file instead of hardcoded, remember last image type. Commit.
  • Increase the flexablilty of importing captions. See issue #1361. Commit.
  • Unbreak slideshow animation. Default animation are still quite bad, we could improve or allow custom. Commit. See bug #438880
  • Fix regression: edit clip always disabled. Commit.
  • Fix concurrency issue breaking effect parameters. Commit.
  • Fix mouse wheel behavior on effect parameters. Commit.
  • Fix guides can be move to position < 0. Commit.
  • Don’t allow editing of current profile to prevent crashes. Commit.
  • Improvements for field order option. Commit.
  • [Project Settings] Fix: open selected profile instead of first. Commit.
  • Seperate HTML and translatable strings. Commit.
  • Add field order option to project settings. Commit.
  • Move audio scrub confif from Kdenlive Settings to Monitor menu, and make it possible with a shortcut. Commit.
  • USe not deprecated KDE_INSTALL_PLUGINDIR variable. Commit.
  • Ensure timeline clips are reloaded on change. Commit.
  • Fix rotation param cannot be copied between keyframes. Commit.
  • Ensure monitors are raised on click in bin or timeline. Commit.
  • Fix “Locate Clip” always hidden. Commit.
  • Don’t allow creating keyframe outside clip after resizing. Commit.
  • On project load, ensure reverse mixes are correctly detected. Commit.
  • Fix timeline thumbs when changing speed of clip. Commit.
  • Fix timeline video thumbs on clips with speed effect. Commit.
  • Fix same track transition resize glitch. Commit.
  • Don’t attempt to proxy mp3 clips with cover art. Commit.
  • [Project Profile Editor] Face lift and refactoring. Commit.
  • Fix audio thumbs of reversed clips. Commit.
  • Fix timeline broken in previous commit. Commit.
  • Fix timeline audio thumbs sometimes not correcty updating. Commit.
  • Transcoding: display info for files requiring transcoding, improve handling of audio only/video only files. Commit.
  • Transcode to edit friendly format: handle audio only and video only clips. Commit.
  • Replace the “Transcode…” option from bin context menu and replace with the option to transcode to an edit friendly format. Commit.
  • Follow MLT consumer properties renaming. Commit.
  • Validate 3D LUT CUBE files to prevent crashing in FFmpeg filter. Commit.
  • Fix tags corrupting audio/video icons in bin. Commit.
  • Fix audio thumb speed not correctly initialized (broke monitor thumbs). Commit.
  • Fix audio thumbs for clips with speed effect. Commit.
  • Fix crash cutting a grouped subtitle. Commit.
  • [Code Gardening] Use qRound instead of +0.5. Commit.
  • Fix 1 pixel offset at some zoom factors. Commit.
  • Fix build with KDE Frameworks < 5.77. Commit.
  • [Effect Stack] Indicate position on drag&drop. Commit.
  • Correctly update add/remove keyframe button on undo/redo and active keyframe on speed change. Commit.
  • Fix possible crash in extract zone. Commit.
  • Fix thumbnails for playlist clips having a different resolution than project profile. Commit.
  • Fix clip target tracks and properties not properly set for first clip when creating a new project. Commit.
  • Fix header comment. Commit.
  • Fix crash updating speed before/after in time remap. Commit.
  • On project close, ensure non loaded clips don’t leak to the new project (part 2). Commit.
  • Fix proxy clips not correctly disabled on rendering. Commit.
  • On project close, ensure non loaded clips don’t leak to the new project. Commit.
  • Add debug info to better understand proxy rendering issue. Commit.
  • Fix sometimes cannot resize clip when there is a 1 frame gap. Commit.
  • Various fixes for remove space in subtitle track. Commit.
  • Time remap: only seek clip monitor if visible. Commit.
  • Fix same track transitions sometimes broken by clip resize. Commit.
  • Fix 1 frame offset in subtitles when removing space. Commit.
  • Fix timeline preview sometimes using old rendered file instead of recreating a preview. Commit.
  • Fix bin clip thumbnails sometimes not created. Commit.
  • Show clip labels as soon as there is one letter width. Commit.
  • Fix marker thumbnail size. Commit.
  • Don’t show clip thumbs on when clip is too small (<16 pixels). Commit.
  • [Titler] Fix: don’t remember outline value with factor 10. Commit.
  • [Bin] Improve/fix ToolBar overflow. Commit.
  • [Monitors] Improve/fix ToolBar overflow. Commit.
  • [Monitors] Improve audio volume widget. Commit.
  • [Wizard] Fix wrong subtitle effect detection. Commit.
  • [Effect UI] Improve/fix ToolBar overflow. Commit.
  • Missing change from last commit (fix remove space). Commit.
  • Fix “remove space” not working on 1 frame space. Commit.
  • Only create proxy clips automatically if requested. Commit.
  • Fix audio wave for non stereo clips. Commit.
  • Remove configureToolbar hack for >= KF 5.91. Commit.
  • [Titler] Fix text outline width steps (does only seem to support int). Commit.
  • Now that everything is fixed in Craft, use the new KNS dialog on windows. Commit.
  • Fix qml binding loop warning. Commit.
  • Add option in timeline preview menu to render preview using original clips, not proxies (disabled by default). Commit.
  • Add Set zone in/out to timeline ruler context menu. Commit.
  • [flatpak] update frei0r for alpha render fix. Commit.
  • Fix clip thumbnails extending past clip length. Commit.
  • Add missing vertical spacer. Commit.
  • Fix adjust to original size using proxy resolution. Commit.
  • Try to fix ghost icons on Windows. Commit.
  • Major speedup in audio thumbs drawing on high zoom levels. Commit.
  • [Nightly Flatpak] Update to ffmpeg 5.0. Commit.
  • Fix clip name not scrolling anymore. Commit.
  • Fix unusable bin icon for audio/video drag. Commit.
  • Don’t update dock title bars several times on layout change. Commit.
  • “Add Marker/Guide quickly” allow shortcut * only on keypad (otherwise *. Commit. Fixes bug #434411
  • Fix action name: “Save Timeline Zone to Bin” is not what it does. Commit. Fixes bug #436386
  • Fix Wayland crash on layout switch. Commit.
  • Minor optimization for audio thumbs drawing. Commit.
  • [Packaging Type] Try to detect snap automatically. Commit.
  • L10n improvements by @bellaperez: *.ui files. Commit.
  • L10n improvements by @bellaperez: Misc. Commit.
  • Show details of external proxy profile in project settings. Commit.
  • Fix .ass subtitle files not correctly read. Commit.
  • Ensure processes are in the path before starting an executable. Commit.
  • Fix timeline keyframes sometimes disappearing from view. Commit.
  • Fix wrong comparison of current settings and settings stored in the project settings dialog. Commit.
  • Fix sometimes cannot move grouped clip right when only 1 empty frame. Commit.
  • Clip stabilize: keep track of bin clip rotation. Commit.
  • Render at preview resolution: store in project file instead of global config. Commit.
  • When saving effect, show it under its name, not id in effect list. Commit.
  • Fix fade effects not correctly saved or pasted. Commit.
  • Fix vp8 with alpha render profile. Commit.
  • Fix clip monitor allowing seek past clip length with transparency background enabled. Commit.
  • Rendering: add option to render at preview resolution for faster preview rendering. Commit.
  • Fix green tint on first image extract. Commit.
  • Show package type in about dialog. Commit.
  • Improve Sandbox detection, use dedicated config files for sandbox packages. Commit.
  • Timeline preview: use compact format when calling renderer. Commit.
  • Refactor Kdenlive Settings. Commit.
  • Minor cleanup of add marker ui. Commit.
  • Ensure thumbnail preview profile is not changed by clip resolution. Commit.
  • More alpha render quality fixes. Commit.
  • PACKAGING CHANGE!! Introduce PACKAGE_TYPE envvar. Commit.
  • Try to improve alpha render quality. Commit. Fixes bug #436879. Fixes bug #430093. Fixes bug #357153
  • When transoding variable fps clips, ensure we keep all streams. Commit.
  • Fix recently introduced memory leak. Commit.
  • Fix MLT’s build for nightly AppImage following a recent MLT change. Commit.
  • Fix alpha render and add utvideo. Commit. Fixes bug #448010. See bug #436879
  • Timeline preview: when saving, store rendered chunks in a more clever list (like 0-1700 instead of listing each 0,25,50,75,… until 1700). Commit.
  • Show number of Bin clips / selected clip in status bar, fix rename / focus clip in icon view. Commit.
  • Change defualt shortcut or “Multitrack View” to F12. Commit.
  • Fix freeze trying to drag a clip that was just added to Bin. Commit.
  • Append fps to clip name when transcoding a variable fps clip. Commit.
  • Add recent MLT rotate feature to easily rotate a video clip from clip properties. Commit.
  • Fix timeline ruler not working after effect drop in some circumstances. Commit.
  • Fix possible crash on undo/redo transcoding. Commit.
  • Warn user when trying to render an empty timeline, some widget cleanup. Commit.
  • Fix various bugs in timeremap (keyframes random move, crashes). Commit.
  • [otio] In case of an error check if it is due to the installed version. Commit.
  • Add default shortcut to “Multitrack view”: Shift+0. Commit.
  • [OTIO] Differentiate between read and write adapters. Commit. Fixes bug #448318
  • Add python interface. Commit. Fixes bug #423083
  • Time Remap: don’t allow keyframe after last frame of source clip. Commit.
  • Timeremap should only be available on clips without B frames. Propose transcoding otherwise. Commit.
  • Show warnings if avfilter, especially avfilter.subtitles was not found. Commit.
  • [Setup Wizard] Show codes if there are only info messages, fix doc link. Commit.
  • Protect timeline preview list with mutex. Commit.
  • Fix minor typo. Commit.
  • At least a little bit more order in configure shortcuts dialog. Commit.
  • Fix slideshow duration not updated on profile change. Commit.
  • Fix save path for custom render profiles on Windows following commit 90b1e4. Commit.
  • Fix detection of missing timeline preview chunks on opening. Commit.
  • Don’t attempt to create audio thumbs if thumbs are disabled. Commit. Fixes bug #448304
  • Neutral background, separator and tooltip for tool name in statusbar. Commit.
  • Speedup loading of projects with timeline preview. Commit.
  • Add some default LUT files. Commit.
  • Revert fileWatcher to private one, as the global app sometimes messes with us (when opening a file dialog). Commit.
  • Fix extract frame on Windows (also used for Titler and scopes). Commit.
  • We still need the magic lantern icon. Commit.
  • Reducing to compatible subset for licensing consistency. Commit.
  • Fix dragging a composition sometimes moved it to the beginning of the clip. Commit.
  • Use a SPDX standard license identifier in Appstream data. Commit. Fixes bug #448134
  • Make it easier to drag a same track composition (include snapping point). Commit.
  • Fix inconsistencies in profile switch check. Commit.
  • Fix loading progress not disappearing and incorrect “clip already exists” message. Commit.
  • Fix bin accepting invalid clips. Commit.
  • Update build MLT >= 7.0.0. Commit.
  • When adding many clips to a project, show loading progress in status bar. Commit.
  • Deprecate and fallback icons and disable by default. Commit.
  • Fix regression after 8f445516159a06654be649d7c70a2aae9788f071. Commit.
  • Fix freeze trying to change profile while load tasks were running. Commit.
  • FIx after 7ef5187fe2d6448fc4c2ae29df6e861ddf7de41c. Commit.
  • Fix crash on layout change. Commit.
  • Fix cleanup regression (disappearing titles). Commit.
  • Fix tab widget tooltips containing ampersand. Commit. See bug #447825
  • Partly Revert 19b9a0b7 since it caused trouble. Commit.
  • Fix shortcuts sometimes broken with fullscreen monitor. Commit.
  • Qml required property is not supported in Qt < 5.15. Commit.
  • Remove unused code for codec check in wizard. Commit.
  • Add missing license headers, fix some existing. Commit.
  • Some clean up of unused code. Commit.
  • Fix bin clip not correctly reloaded on profile change, causing missing audio thumbs. Commit.
  • Massive speedup on project load (at least when working on nfs filesystem). Commit.
  • Switch to global fileWatcher. Commit.
  • Edit friendly transcoding: add option to disable autorotate during transcoding. Commit.
  • [Monitors] Ctrl+Wheel instead of Ctrl+Shift+Wheel. Commit.
  • [Clip Monitor] Fix flicker on hover of “In Point” / “Out Point” Label. Commit.
  • Refactor monitor tool bars (Pt. 1). Commit.
  • Referactoring: Move ZoomBar to a generic component. Commit.
  • Minor optimization on project load (don’t unnecessarily request a frame). Commit.
  • [Timeline Zoombar] More improvements. Commit.
  • [Timeline Zoombar] Improvments. Commit.
  • Fix shortcuts sometimes broken on fullscreen monitor. Commit.
  • Fix build after last commit. Commit.
  • [Splash] Add KDE Branding, cleanup unused files. Commit.
  • Fix minor typo. Commit.
  • Fix minor typos. Commit.
  • Fix audio thumbs not created after profile change. Commit.
  • Fix window title using custom path instead of profile description. Commit.
  • Fix compilation warnings (function type compatibility). Commit.
  • Automatically offer to transcode variable frame rate clips. Commit.
  • Happy New Year Kdenlive! (Update AboutData Copyright Year). Commit.
    • Check for variable frame rate clips and propose transcoding (WIP). Commit.
  • [Titler] Add some tool tips for select actions. Commit.
  • Add xml ui for audiolevelgraph effect and other xml format fixes. Commit.
  • Fix compilation with KF5 < 5.89. Commit.
  • Fix multiple bins should always stay tabbed together. Commit.
  • Add JP2 image format. Commit.
  • Minor improvements for OTIO handling. Commit.
  • Update user manual link for dasiam instructions. Commit.
  • Fix profile corruption. Commit.
  • Fix possible crash working with placeholder clips with speed effect. Commit.
  • Include clip markers and effect params in test hash function for better regression tracking. Commit.
  • Ability to make image seqiences from raw photos. Commit.
  • Move default lut value to proper function. Commit.
  • CppCheck fixes Pt. 4. Commit.
  • CppCheck fixes Pt. 3. Commit.
  • CppCheck fixes Pt. 2. Commit.
  • Fix last commit always resetting lut file to first installed one on project load. Commit.
  • Ensure lut effect is initialized with a file when added. Commit.
  • Update file test, add a timeline hash function to check if a document is identical before / after save. Commit.
  • Fix inconsistencies in subtitle model leading to broken (uneditable) items. Commit.
  • Remove unused/duplicate code. Commit.
  • Fix some compiler and CppCheck warnings. Commit.
  • Re-use NegQColor. Commit.
  • Fix build with fuzzing. Commit.
  • Attempt to fix threading test crash. Commit.
  • [nightly flatpak] update dependencies. Commit.
  • Add tests to prevent project corruption on color/title/image clip resize as happened in 21.08.3. Commit.
  • Use AppLocalDataLocation on all plattform to finde KNS items. Commit.
  • Improve urllistwidget for effects. Commit.
  • Fix compile failure after last merge. Commit.
  • Make it possible to run composition tests independent from each other. Commit.
  • Fix tests. Instead of returning random profile, use dv_pal when no config file is found. Commit.
  • Uptade frei0r.scale0tilt.xml with Scale X and Y parameters now animated. Commit.
  • Make it possible to enable/disable track with a shortcut. Commit. Fixes bug #440181
  • Clean up code after 9aaf43a1. Commit.
  • Install templates to “kdenlive/effect-templates”. Commit.
  • Add option to download effect templates from store.kde.org. Commit.
  • Make it possible to export custom effect xmls. Commit.
  • Add two template effects authored by @massimostella. Commit.
  • Add infrastruture for template effects. Commit.
  • Fix display of timeline usage in clip monitor. Commit.
  • Show timeline usage in clip monitor. Commit.
  • Add some tests for spacer operations. Commit.
  • Require at least CMake 3.16. Commit.
  • Add UI for the frei0r_transparency effect. Commit.
  • Attempt to fix slideshow detection on Windows. Commit.
  • Add a position widget to mix parameters to allow sliding the mix. Commit.

The post Kdenlive 22.04 released appeared first on Kdenlive.

Categories: FLOSS Project Planets

Mike Driscoll: PyDev of the Week: Jyotika Singh

Planet Python - Mon, 2022-05-02 08:30

The PyDev of the Week this week is Jyotika Singh (@JyotikaSingh_). Jyotika is the maintainer of pyAudioProcessing and a speaker at multiple conferences. You can check out what Jyotika is up to by going to her GitHub profile.

Let's spend a few minutes getting to know Jyotika better!

Can you tell us a little about yourself (hobbies, education, etc): 

I work as the Director of Data Science at Placemakr and volunteer as a mentor at Data Science Nigeria and Women Impact Tech. I actively participate in conferences and webinars to share my knowledge and experiences with the Python and Data Science community, and with students aspiring to a career in software development and data science. As part of my research, I have been granted multiple patents in data science, algorithms, and marketing optimization techniques.

I graduated with a Master's in Science degree from the University of California, Los Angeles (UCLA), specializing in Signals and Systems.

When not engaged in technology and coding, I enjoy sketching, painting, playing musical instruments, and at times enjoy my evenings at the beach.

Why did you start using Python? 

The first time I used Python was to work on an assignment on data scraping during the course of my MS in 2015. While the course didn't require me to use Python, I chose to take on the new language given the ease of writing and availability of ML libraries which I was just beginning to look into. As my MS progressed, I kept learning more of Python and got hooked onto its ease and functionality, and compatibility with other tools.

What other programming languages do you know and which is your favorite? 

I have primarily worked with MATLAB, C, Java, R, Golang, and Python. Python has hands-down been my favorite for a while now.

What projects are you working on now? 

For the next few months, I'm working on a few different projects.

  1. Pricing recommendation and optimization models
  2. Content-based recommendation systems
  3. Consumer review analysis and classification
  4. A reinforcement learning-based model for ROI optimization

Furthermore, I'm in the process of writing a book on industrial applications and implementations of Natural Language Processing.

How did you decide to write a book about NLP with Python? 

There is a known gap between what data science graduates master versus the needs of a data scientist in the industry. The majority of the projects start with the availability of data in a state that does not require much thought behind where the data comes from or what data is required. The most commonly available learning resources have attributes far from real-world applications.

For individuals in software development, technology, product management, or those that are new to data science or Natural Language Processing, learning about what to solve and how to proceed can be a challenging problem. Keeping this in mind, I decided to write this book that explains the application across 15 industry verticals. It is set to dig into practical implementations of many popular applications and contains actual code examples. This book will guide users to build applications with Python and highlight the reality of data problems and solutions in the real world.

Which Python libraries are your favorite (core or 3rd party)? 

os, collections, pandas, numpy, sklearn, pytorch, tensorflow, keras, spacy, nltk, matplotlib, and my own pyAudioProcessing.

How did the pyAudioProcessing library come about? 

I was working on a unique audio classification problem. Given the popularity of ML tooling in Python, I was looking to build my audio models in Python. I noticed a large gap between the research and development happening in MATLAB versus the state of 3rd party tooling in Python around audio.

Different types of non-numeric data have different feature formation techniques that work well for numerically representing the data in a meaningful way. An example for text data would be TF-IDF. Similarly, audio data has a completely different and its own set of feature formation techniques that represent the information in a numerical sense. I took it upon myself to mathematically construct audio features from raw audio in Python, and decided to open-source my work while recognizing the need. This gave rise to PyAudioProcessing. Today, you can extract features such as GFCC, MFCC, spectral features, and chroma features using PyAudioProcessing. Its integration with other 3rd party libraries helps extract audio visualizations, audio format conversion, build audio classification models using sklearn models, and use off-the-shelf audio classification models for some common tasks.

What challenges do you face as a maintainer of a Python package? 

Given a packed schedule with my full-time job, research, conferences, and mentorship volunteering, the most challenging bit is taking the time out for continuous development and keeping the library up-to-date with new research, needs, features, and compatibility with the latest Python releases. Contributors are always welcome!

Is there anything else you’d like to say? 

I would like to thank all the people who open-source their work that the community is able to leverage for their personal and professional projects. Also, a huge shout out to people who volunteer and make efforts to share their knowledge and findings via events organized by the Python and Data community.

I'm on Twitter at jyotikasingh_, follow me to catch my latest talks, work, and findings.

Thanks for doing the interview, Jyotika!

The post PyDev of the Week: Jyotika Singh appeared first on Mouse Vs Python.

Categories: FLOSS Project Planets

Tryton News: Release of python-sql 1.4.0

Planet Python - Mon, 2022-05-02 06:43

We are proud to announce the release of the version 1.4.0 of python-sql.

python-sql is a library to write SQL queries in a pythonic way. It is mainly developed for Tryton but it has no external dependencies and is agnostic to any framework or SQL database.

In addition to bug-fixes, this release contains the following improvements:

  • Use unittest discover
  • Use only column name for INSERT and UPDATE
  • Add escape to Like operators
  • Add default literal ‘*’ expression to Count
  • Add support for Python 3.10

python-sql is available on PyPI: python-sql · PyPI

1 post - 1 participant

Read full topic

Categories: FLOSS Project Planets

Tryton News: Releaset of Relatorio 0.10.1

Planet Python - Mon, 2022-05-02 06:39

We are proud to announce the release of Relatorio version 0.10.1.

Relatorio is a templating library mainly for OpenDocument using also OpenDocument as source format.

This is a bug-fix release which:

  • Add support for Python 3.10
  • Support directive applying to itself
  • Keep tail of directive
  • Use unittest discover

The package is available at https://pypi.org/project/relatorio/0.10.1/
The documentation is available at https://relatorio.readthedocs.io/en/0.10.1/

1 post - 1 participant

Read full topic

Categories: FLOSS Project Planets

Podcast.__init__: Accelerate Your Machine Learning Experimentation With Automatic Checkpoints Using FLOR

Planet Python - Mon, 2022-05-02 06:14
The experimentation phase of building a machine learning model requires a lot of trial and error. One of the limiting factors of how many experiments you can try is the length of time required to train the model which can be on the order of days or weeks. To reduce the time required to test different iterations Rolando Garcia Sanchez created FLOR which is a library that automatically checkpoints training epochs and instruments your code so that you can bypass early training cycles when you want to explore a different path in your algorithm. In this episode he explains how the tool works to speed up your experimentation phase and how to get started with it.Summary

The experimentation phase of building a machine learning model requires a lot of trial and error. One of the limiting factors of how many experiments you can try is the length of time required to train the model which can be on the order of days or weeks. To reduce the time required to test different iterations Rolando Garcia Sanchez created FLOR which is a library that automatically checkpoints training epochs and instruments your code so that you can bypass early training cycles when you want to explore a different path in your algorithm. In this episode he explains how the tool works to speed up your experimentation phase and how to get started with it.

  • Hello and welcome to Podcast.__init__, the podcast about Python’s role in data and science.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • Your host as usual is Tobias Macey and today I’m interviewing Rolando Garcia about FLOR, a suite of machine learning tools for hindsight logging that lets you speed up model experimentation by checkpointing training data
  • Introductions
  • How did you get introduced to Python?
  • Can you describe what FLOR is and the story behind it?
  • What is the core problem that you are trying to solve for with FLOR?
    • What are the fundamental challenges in model training and experimentation that make it necessary?
    • How do machine learning reasearchers and engineers address this problem in the absence of something like FLOR?
  • Can you describe how FLOR is implemented?
    • What were the core engineering problems that you had to solve for while building it?
  • What is the workflow for integrating FLOR into your model development process?
  • What information are you capturing in the log structures and epoch checkpoints?
    • How does FLOR use that data to prime the model training to a given state when backtracking and trying a different approach?
  • How does the presence of FLOR change the costs of ML experimentation and what is the long-range impact of that shift?
    • Once a model has been trained and optimized, what is the long-term utility of FLOR?
  • What are the opportunities for supporting e.g. Horovod for distributed training of large models or with large datasets?
  • What does the maintenance process for research-oriented OSS projects look like?
  • What are the most interesting, innovative, or unexpected ways that you have seen FLOR used?
  • What are the most interesting, unexpected, or challenging lessons that you have learned while working on FLOR?
  • When is FLOR the wrong choice?
  • What do you have planned for the future of FLOR?
Keep In Touch Picks Closing Announcements
  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Categories: FLOSS Project Planets

Zato Blog: Integrating with Salesforce in Python

Planet Python - Mon, 2022-05-02 05:31

Salesforce connections are one of the newest additions to Zato 3.2, allowing you to look up and manage Salesforce records and other business data. To showcase it, the article will create a sample Salesforce marketing campaign in a way that does not require the usage of anything else except for basic REST APIs combined with plain Python objects, such as dicts.

If you have not done it already, you can download Zato here.

Basic workflow

The scope of our works will be:

  • Creating Salesforce credentials for our integration project
  • Defining a Salesforce connection in Zato
  • Authoring a service that will map input data to the format that Salesforce expects
  • Creating a Zato REST channel that will be invoked through curl during tests
  • Testing the integration
Creating Salesforce credentials

To be able to create as connection to Salesforce in the next step, we need a few credentials.

In runtime, based on this information, Zato will obtain the necessary authentication and authorization tokens itself, which means that you will only focus on the business side of the integrations, not on the low-level aspects of it.

The process of obtaining the credentials needs to be coordinated with an administrator of your organization. To assist in that, the screenshots below explain where to find them.

The credentials are:

  • Username and password
  • Consumer key
  • Consumer secret

The username and password are simply the same credentials that can be used to log in to Salesforce:

Consumer key and secret are properties of a connected app - this is a term that Salesforce uses for API clients that invoke its services. If you are already an experienced Salesforce REST API user, you may know the key and secret under their aliases of “client_id” and “client_secret” - these are the same objects.

Note that when a connected app already exists and you would like to retrieve the key and secret, they will be available under the “View” menu option for the app, not under “Edit” or “Manage”.

Defining a Salesforce connection in Zato

With all the credentials in place, we can create a new Salesforce connection in Zato Dashboard, as below.

Authoring an integration service in Python

Above, we created a connection definition that lets Zato obtain session tokens and establish connections to Salesforce. Now, we can create an API service that will make use of such connections.

In the example below, we are using the POST REST method to invoke an endpoint that creates new Salesforce campaigns. In your own integrations, you can invoke any other Salesforce endpoint, using any REST method as needed, by following the same pattern, which is, create a model with input fields, build a Python dict for the request to Salesforce, invoke it and map all the required from the response from Salesforce to that which your own service returns to its own callers.

Note that we use a datamodel-based SimpleIO definition for the service. Among other things, although we are not going to do it here, this would let us offer definitions for this and other services.

# -*- coding: utf-8 -*- # stdlib from dataclasses import dataclass # Zato from zato.server.service import Model, Service # ########################################################################### if 0: from zato.server.connection.salesforce import SalesforceClient # ########################################################################### @dataclass(init=False) class CreateCampaignRequest(Model): name: str segment: str # ########################################################################### @dataclass(init=False) class CreateCampaignResponse(Model): campaign_id: str # ########################################################################### class CreateCampaign(Service): class SimpleIO: input = CreateCampaignRequest output = CreateCampaignResponse def handle(self): # This is our input data input = self.request.input # type: CreateCampaignRequest # Salesforce REST API endpoint to invoke - note that Zato # will add a prefix to it containing the API version. path = '/sobjects/Campaign/' # Build the request to Salesforce based on what we received request = { 'Name': input.name, 'Segment__c': input.segment, } # .. create a reference to our connection definition .. salesforce = self.cloud.salesforce['My Salesforce Connection'] # .. obtain a client to Salesforce .. with salesforce.conn.client() as client: # type: SalesforceClient # .. create the campaign now .. response = client.post(path, request) # .. and return its ID to our caller. self.response.payload.campaign_id = response['id'] # ########################################################################### Creating a REST channel

Note that we assign HTTP Basic Auth credentials to the channel. In this manner, it is possible for clients of this REST channel to authenticate using a method that they are already familiar which simplifies everyone’s work - it is Zato that deals with how to authenticate against Salesforce whereas your API clients use the ubiquitous HTTP Basic Auth method.


The last step is to invoke the newly created channel:

$ curl http://api:password@localhost:17010/api/campaign/create -d '{"name":"Hello", "segment":"123"}' {"campaign_id":"8901Z3VHXDTebEJWs"} $

That is everything - you have just integrated with Salesforce and exposed a REST channel for external applications to integrate with!

Next steps
  • Start the tutorial to learn how to integrate APIs and build systems. After completing it, you will have a multi-protocol service representing a sample scenario often seen in banking systems with several applications cooperating to provide a single and consistent API to its callers.

  • Visit the support page if you need assistance.

  • Para aprender más sobre las integraciones de Zato y API en español, haga clic aquí.

  • Pour en savoir plus sur les intégrations API avec Zato en français, cliquez ici.

Categories: FLOSS Project Planets

John Ludhi/nbshare.io: PySpark GroupBy Examples

Planet Python - Mon, 2022-05-02 03:39
PySpark GroupBy Examples

In this notebook, we will go through PySpark GroupBy method. For this exercise, I will be using following data from Kaggle...

If you don't have PySpark installed, install Pyspark on Linux by clicking here.

In [ ]: from pyspark.sql.functions import sum, col, desc, avg, round, count import pyspark.sql.functions as F from pyspark.sql import SparkSession from pyspark.sql.types import * spark = SparkSession \ .builder \ .appName("Purchase") \ .config('spark.ui.showConsoleProgress', False) \ .getOrCreate()

Let us look at the data first.

In [2]: df = spark.read.csv( "/home/notebooks/kz.csv", header=True, sep=",") #show 3 rows of our DataFrame df.show(3) +--------------------+-------------------+-------------------+-------------------+--------------------+-------+------+-------------------+ | event_time| order_id| product_id| category_id| category_code| brand| price| user_id| +--------------------+-------------------+-------------------+-------------------+--------------------+-------+------+-------------------+ |2020-04-24 11:50:...|2294359932054536986|1515966223509089906|2268105426648170900| electronics.tablet|samsung|162.01|1515915625441993984| |2020-04-24 11:50:...|2294359932054536986|1515966223509089906|2268105426648170900| electronics.tablet|samsung|162.01|1515915625441993984| |2020-04-24 14:37:...|2294444024058086220|2273948319057183658|2268105430162997728|electronics.audio...| huawei| 77.52|1515915625447879434| +--------------------+-------------------+-------------------+-------------------+--------------------+-------+------+-------------------+ only showing top 3 rows In [3]: df.columns Out[3]: ['event_time', 'order_id', 'product_id', 'category_id', 'category_code', 'brand', 'price', 'user_id']

This is transaction data.

PySpark Groupby Count

Let us count number of unique transactions by categories.

In [4]: df.groupBy(['category_code']).count().show(5) +----------------+-----+ | category_code|count| +----------------+-----+ | 13.87|11075| | 350.67| 5| |computers.ebooks| 884| | 98.59| 2| | 3.89| 6997| +----------------+-----+ only showing top 5 rows

PySpark groupby and count can be run on multiple columns.

In [5]: df.groupBy(['category_code','brand']).count().show(5) +--------------------+-------------------+-----+ | category_code| brand|count| +--------------------+-------------------+-----+ |electronics.smart...| oppo|36349| |appliances.enviro...| airline| 52| |computers.periphe...| sanc| 584| |appliances.enviro...| insight| 11| | 11.55|1515915625481232307| 1| +--------------------+-------------------+-----+ only showing top 5 rows PySpark drop null follow by GroupBy In [6]: dfg = df.dropna().groupBy(['category_code']) In [7]: dfg.count().show(2) +--------------------+-----+ | category_code|count| +--------------------+-----+ | computers.ebooks| 398| |computers.periphe...| 3053| +--------------------+-----+ only showing top 2 rows PySpark GroupBy and Aggregate

Most of the times, groupby is followed by aggregate method. Let us say we want to find the average price for each category. Here is how it can be done.

In [8]: df.dropna().groupBy(['category_code']).agg({'price':'avg'}).show(5) +--------------------+------------------+ | category_code| avg(price)| +--------------------+------------------+ | computers.ebooks| 199.6687185929649| |computers.periphe...| 71.94989518506395| |construction.tool...| 18.2120273065784| |appliances.kitche...|43.298406940063074| |electronics.video...| 401.3619130434783| +--------------------+------------------+ only showing top 5 rows

Note, pyspark has named the average price column to avg(price). We can rename the column name after aggregate method with withColumnRenamed method.

In [9]: df.dropna().groupBy(['category_code']).agg({'price':'avg'}).withColumnRenamed("avg(price)", "price").show(5) +--------------------+------------------+ | category_code| price| +--------------------+------------------+ | computers.ebooks| 199.6687185929649| |computers.periphe...| 71.94989518506395| |construction.tool...| 18.2120273065784| |appliances.kitche...|43.298406940063074| |electronics.video...| 401.3619130434783| +--------------------+------------------+ only showing top 5 rows

Another way to rename the column in pyspark is using alias method.

In [10]: df.dropna().groupBy(['category_code']).agg(avg('price').alias("avg_price")).show(3) +--------------------+-----------------+ | category_code| avg_price| +--------------------+-----------------+ | computers.ebooks|199.6687185929649| |computers.periphe...|71.94989518506395| |construction.tool...| 18.2120273065784| +--------------------+-----------------+ only showing top 3 rows Pyspark Multiple Aggregate functions

We can also run multiple aggregate methods after groupby. Note F.avg and F.max which we imported above from pyspark.sql.
import pyspark.sql.functions as F

In [11]: df.dropna().groupBy(['category_code']).agg(F.avg('price'),F.max('price')).show(2) +--------------------+------------------+----------+ | category_code| avg(price)|max(price)| +--------------------+------------------+----------+ | accessories.bag| 20.63646942148758| 97.20| |accessories.umbrella|110.71249999999998| 99.28| +--------------------+------------------+----------+ only showing top 2 rows

We can rename the multiple columns using toDF() method as shown below.

In [12]: Data_list = ["category_code","avg_price","max_price"] df.dropna().groupBy(['category_code']).agg(F.avg('price'),F.max('price')).toDF(*Data_list).show(2) +--------------------+------------------+---------+ | category_code| avg_price|max_price| +--------------------+------------------+---------+ | accessories.bag| 20.63646942148758| 97.20| |accessories.umbrella|110.71249999999998| 99.28| +--------------------+------------------+---------+ only showing top 2 rows

or we can use alias method this way...

In [13]: df.dropna().groupBy(['category_code']).agg(avg('price').alias("avg_price"),F.max('price').alias("max_price")).show(3) +--------------------+------------------+---------+ | category_code| avg_price|max_price| +--------------------+------------------+---------+ | accessories.bag| 20.63646942148758| 97.20| |accessories.umbrella|110.71249999999998| 99.28| | apparel.costume|21.384999999999998| 27.75| +--------------------+------------------+---------+ only showing top 3 rows PySpark GroupBy follow by Aggregate and Sort Method

Let us sort the table by max_price.

In [14]: df.dropna().groupBy(['category_code']).agg(F.avg('price'),F.max('price')).toDF(*Data_list).sort('max_price').show(2) +--------------+------------------+---------+ | category_code| avg_price|max_price| +--------------+------------------+---------+ | kids.swing| 115.72| 115.72| |apparel.tshirt|21.384516129032253| 23.13| +--------------+------------------+---------+ only showing top 2 rows PySpark GroupBy follow by Aggregate and Filter method

We can filter results using Filter method. Below code filters the categories which have average price greater than 500.

In [15]: dfg = df.dropna().groupBy(['category_code']).agg(F.avg('price').alias("avg_price")) dfg.filter(dfg.avg_price> 500).show(4) +--------------------+-----------------+ | category_code| avg_price| +--------------------+-----------------+ |electronics.camer...| 670.243984962406| |construction.tool...|513.4461206896547| | computers.notebook|571.6449383765361| +--------------------+-----------------+ Conclusion

PySpark GroupBy is very powerful method to do data analysis. I hope above examples gave you enough to get started on PySpark GroupBy. Please email me if you want me to add more examples on PySpark Groupby.

Categories: FLOSS Project Planets

Thomas Koch: Missing memegen

Planet Debian - Sun, 2022-05-01 14:17
Posted on May 1, 2022 Tags: debian, free software, life

Back at $COMPANY we had an internal meme-site. I had some reputation in my team for creating good memes. When I watched Episode 3 of Season 2 from Yes Premier Minister yesterday, I really missed a place to post memes.

This is the full scene. Please watch it or even the full episode before scrolling down to the GIFs. I had a good laugh for some time.

With Debian, I could just download the episode from somewhere on the net with youtube-dl and easily create two GIFs using ffmpeg, with and without subtitle:

ffmpeg -ss 0:5:59.600 -to 0:6:11.150 -i Downloads/Yes.Prime.Minister.S02E03-1254485068289.mp4 tmp/tragic.gif ffmpeg -ss 0:5:59.600 -to 0:6:11.150 -i Downloads/Yes.Prime.Minister.S02E03-1254485068289.mp4 \ -vf "subtitles=tmp/sub.srt:force_style='Fontsize=60'" tmp/tragic_with_subtitle.gif

And this sub.srt file:

1 00:00:10,000 --> 00:00:12,000 Tragic.

I believe, one needs to install the libavfilter-extra variant to burn the subtitle in the GIF.







The Premier Minister just learned, that his predecessor, who was about to publish embarassing memories, died of a sudden heart attack:

I can’t actually think of a meme with this GIF, that the internal thought police community moderation would not immediately take down.

For a moment I thought that it would be fun to have a Meme-Site for Debian members. But it is probably not the right time for this.

Maybe somebody likes the above GIFs though and wants to use them somewhere.

Categories: FLOSS Project Planets

#! code: Drupal 9: Using Lazy Builders

Planet Drupal - Sun, 2022-05-01 13:40

The lazy builder API in Drupal allows the creation of highly dynamic content within a render array without having to disable the cache for the entire render array or the page the content is attached to. What this means in real terms is that the initial render array can be cached quite heavily, but lazy builders allows for additional rendering to be done in after the initial rendering pass to generate content.

There are several reasons why lazy builders might be useful. The most useful reasons are to present some highly personalised or dynamic content to a user, but you might also want to offset the rendering of some slow code to give it some more fine grained caching that stores the outcome in a cache bucket.

Lazy builders work by using a normal render array, but instead of rendering the content you inject a lazy builder placeholder into the content. A callback method is used to tell Drupal how to inject the content into the placeholder. Then, at a later stage of the rendering process, Drupal will call the callback function and replace the placeholder with the actual content.

If the Big Pipe module is installed then you can also benefit from the page being streamed to the user in parts. This means that if your render function is slow then it won't cause the rest of the page to be slow. The Big Pipe module will add in placeholders to the markup as the render array is generated that are then sent to the browser first. The footer of the page is then rendered and sent separately and JavaScript is used to replace the placeholders with the actual content.

Whilst lazy building doesn't provide you with performance benefits, it can be used in conjunction with Big Pipe to mask those problems so that your page will appear to load quickly.

Read more.

Categories: FLOSS Project Planets