Osamu Aoki: exim4 configuration for Desktop (better gmail support)

Planet Debian - Tue, 2020-08-04 11:03
Since gmail rewrites "From:" address now (2020) and keep changing access limitation, it is wise not  to use it as smarthost any more.  (If you need to access multiple gmail addresses from mutt etc, use esmtp etc.)
---For most of our Desktop PC running with stock exim4 and mutt, I think sending out mail is becoming a bit rough since using random smarthost causes lots of trouble due to the measures taken to prevent spams.

As mentioned in Exim4 user FAQ , /etc/hosts should have FQDN with external DNS resolvable domain name listed instead of localdomain to get the correct EHLO/HELO line.  That's the first step.

The stock configuration of exim4 only allows you to use single smarthost for all your mails.  I use one address for my personal use which is checked by my smartphone too.  The other account is for subscribing to the mailing list.  So I needed to tweak ...

Usually, mutt is smart enough to set the From address since my .muttrc has

# Set default for From: for replyes for alternates.
set reverse_name

So how can I teach exim4 to send mails depending on the  mail accounts listed in the From header.

For my gmail accounts, each mail should be sent to the account specific SMTP connection matching your From header to get all the modern SPAM protection data in right state.  DKIM, SPF, DMARC...  (Besides, they overwrite From: header anyway if you use wrong connection.)

For my debian.org mails, mails should be sent from my shell account on people.debian.org so it is very unlikely to be blocked.  Sometimes, I wasn't sure some of these debian.org mails sent through my ISP's smarthost are really getting to the intended person.

To these ends, I have created small patches to the /etc/exim4/conf.d files and reported it to Debian BTS: #869480 Support multiple smarthosts (gmail support).  These patches are for the source package.

To use my configuration tweak idea, you have easier route no matter which exim version you are using.  Please copy and read pertinent edited files from my github site to your installed /etc/exim4/conf.d files and get the benefits.
If you really wish to keep envelope address etc. to match From: header, please rewite agressively using the From: header using eddited rewrite/31_exim4-config_rewriting as follows:
*@+local_domains "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
# identical rewriting rule for /etc/mailname
*@ETC_MAILNAME "${lookup{${local_part}}lsearch{/etc/email-addresses}\
                   {$value}fail}" f
* "$h_from:" Frs

So far its working fine for me but if you find bug, let me know.

Categories: FLOSS Project Planets

Mobomo: Drupal vs WordPress: Which is Better for Your Business?

Planet Drupal - Tue, 2020-08-04 10:00

Like many developers, some of our first websites were built on the backbones of WordPress. It’s the hyper-popular king of content management systems. It has name recognition, an overflowing user base, and plenty of third-party integrations that help cut your development time. But, over the years, we’ve migrated almost exclusively to Drupal. So why did we switch? What is it about Drupal that leaves developers drooling? And why would anyone pick Drupal — which has around 1.3 million users — over WordPress —which has over 400 million users? Today, we’re going to compare David to Goliath. Why is Drupal, the third most active CMS behind WordPress and Joomla, a good choice for businesses looking to build a refreshing, impactful, and feature-rich website?


By far, the most significant difference between WordPress and Drupal is the overall development need. WordPress is simple. There are hundreds of thousands of third-party plugins that you can leverage to build an entire website with virtually no coding or developing knowledge. And, that’s the single biggest reason that WordPress is so massive. Anyone can build a WordPress site. It’s easy. Drupal requires development. If you want to build a Drupal website, you’re going to have to hire some developers. So, naturally, Drupal has fewer overall users. But, it’s essential to make that distinction. Drupal is built for businesses, public entities, and enterprises. WordPress is built for your everyday website. It’s important to keep this main difference in mind. It’s this difference that resonates throughout these core pillars. And, it’s this core difference that creates pros and cons for each platform.


We consider security, flexibility, and scalability to be the three primary pillars of a CMS. An amazing designer can make a fantastic template or theme regardless of the CMS. And ease-of-use is relative to your plugins/modules, familiarity with the platform, and overall development capabilities. So those are both highly subjective. Security, flexibility, and scalability aren’t subjective; they are what they are.


WordPress has a security problem. Alone, WordPress accounts for 90% of all hacked websites that use a CMS. There’s a tradeoff that comes with leveraging third-party plugins to build websites. You increase your threat landscape. WPScan Vulnerability Database shows 21,675 vulnerabilities in WordPress’s core and with third-party plugins. This security vulnerability issue has been an ongoing headache for WordPress from the start. If we do a play-by-play, year-over-year of WordPress’s history, we see an ongoing and consistent security issue:

  • 2013: 70% of the top 40,000 most popular WordPress websites were vulnerable to hackers
  • 2014: SoakSoak compromises +100,000 websites, a massive DDOS attack hits 160,000 websites, and All In One SEO Pack puts +19 million sites at risk.
  • 2015: A core vulnerability puts millions of websites at risk, Akismet opens millions of websites to hackers, and YoastSEO puts over 14 million websites in hackers’ crosshairs.
  • 2016: At this point, millions of hacks are happening every week across plugins. Check out this WordFence weekly update during this period.
  • 2017: The hacks continue. The average small business website using WordPress is attacked 44 times a day at this point, and WordPress websites are 2x more likely to be hacked than other CMS.

The list goes on. Year-over-year, more vulnerabilities happen across WordPress. And this is an important point. WordPress has subpar security by design. It’s the tradeoff they made to build an ecosystem that doesn’t require development. We aren’t saying that the core of WordPress is inherently security-stripped. It’s not. But, given the scale, scope, and third-party-fanatic nature of the platform, it’s weak on security by nature. Drupal, on the other hand, is the opposite. Websites require development time, each website is customized to the user, and building a website takes time and patience. The tradeoff is better security. Drupal has built-in enterprise-scale security, and you don’t rely on a hotchpotch of third-party applications to build your website’s functionality. There’s a reason that NASA, the White House, and other government entities use (or used) Drupal. It has better security. We want to take a second to make the distinction. WordPress has a secure core. We would argue that Drupal has a more secure core. But the difference isn’t massive. WordPress’s security vulnerabilities are a product of its reliance on third-party applications to make a functional website.


WordPress is more flexible than Drupal to some users. And Drupal is more flexible than WordPress to some users. That may sound complicated. But it comes down to your development capabilities. Drupal has more features than WordPress. Its core is filled with rich taxonomies, content blocks, and unique blocks than WordPress. But, if you aren’t experienced, you probably won’t find and/or use many of these functionalities. On the surface, WordPress has more accessible features. At the core, Drupal is the single most feature-rich CMS on the planet. So, for businesses (especially public entities and larger enterprises), Drupal has a more robust architecture to tackle large-scale projects that have hyper-specific needs. For small businesses and personal website owners, WordPress is easier to use and requires far less development experience to tap into its functionalities, features, and flexibility.


Drupal has better scalability. This one isn’t a competition. Again, this comes down to the dev-heavy nature of the platform. To scale WordPress websites, you add more plugins. To scale Drupal websites, you develop more. There’s a key practical difference here. Drupal modules, taxonomies, and content blocks all exist in the same ecosystem. Each WordPress plugin is its own micro-ecosystem. So, with WordPress, most users are stringing together a ton of third-party ecosystems in an attempt to create one overarching website. Also, Drupal is built for enterprise-scale projects. So there’s backend support and a large landscape of community support around large-scale projects. WordPress is a catch-all CMS that has a little of everything. If WordPress is a Swiss army knife, Drupal is a custom, hand-forged bread knife — explicitly designed to help you scale, slice, and butter larger projects.


At Mobomo, we specialize in Drupal development projects. Our agile-based team of top-level design, development, and support talent can help you launch and scale your website to fit your unique needs. From NASA to Great Minds, we help private and public entities build dreams and execute visions.

Contact us to learn more.

The post Drupal vs WordPress: Which is Better for Your Business? appeared first on .

Categories: FLOSS Project Planets

Real Python: Practical Recipes for Working With Files in Python

Planet Python - Tue, 2020-08-04 10:00

Python has several built-in modules and functions for handling files. These functions are spread out over several modules such as os, os.path, shutil, and pathlib, to name a few. This course gathers in one place many of the functions you need to know in order to perform the most common operations on files in Python.

In this course, you’ll learn how to:

  • Retrieve file properties
  • Create directories
  • Match patterns in filenames
  • Traverse directory trees
  • Make temporary files and directories
  • Delete files and directories
  • Copy, move, or rename files and directories
  • Create and extract ZIP archives

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Mike Driscoll: Is the Python Community Becoming Toxic?

Planet Python - Tue, 2020-08-04 09:31

The Python community is amazing. I started learning Python over 15 years ago and the community was almost always very supportive in helping me figure things out. However, the past few years there seems to have been a shift. I’m not sure if it’s just because Python has grown so much in popularity or if it’s something more basic, such as people becoming more sensitive about things. Whatever it is, the community seems to be heading away from what it once was.

I first started thinking about this during Brett Cannon’s PyCon keynote about his experiences in the open-source community and how we need to be nice to each other. Too many people think they can be rude when requesting features or bug fixes. But he also mentioned that maintainers also need to have a good attitude and not drive away potential new contributors.

A couple months after this keynote was when Guido Van Rossum, creator of the Python language, suddenly retired as the head of Python. At the time, the reason given was that there was so much acrimony and fighting over PEP 572 that he stepped down early.

This year we saw multiple members of the PyTest team drop out of the project.

While Reddit and StackOverflow remain very popular, in my experience I have found them to be difficult to break into. The Reddit Python community, while very large and diverse, is full of trolls and the moderators don’t seem to follow Reddit’s own rules. I personally have had problems simply posting articles on there while others I know have been harassed because their project wasn’t deemed to be “Pythonic” enough. The PySimpleGUI project has been demonized repeatedly there, for example.

I think we can do better. Python is still my favorite language and its community is still a lot of fun. I think we should take note of what’s been happening in our community and make a conscious effort to be nicer to each other. Be mindful when you are writing a bug report or requesting a feature. Most of these projects are run by volunteers in their spare time for free.

The core developers of these projects also need to be kind too though. I remember when I tried to report a bug one time and received a very terse message in return that it was a duplicate or they already knew about the issue. Even experienced developers don’t always know how to search for the proper keywords, especially if they are a new user of that package or technology.

I just wanted to take a moment to bring this topic to light and encourage my readers to think before they write or speak. There’s a real person on the other end who may be having a bad day. Don’t make it worse. In fact, if you can, make it better!

The post Is the Python Community Becoming Toxic? appeared first on The Mouse Vs. The Python.

Categories: FLOSS Project Planets

Announcing Krita’s Scripting School!

Planet KDE - Tue, 2020-08-04 08:13

In 2018, we reintroducted scripting to Krita. Unlike our previous attempts to provide scripting, this time it took off! People are making all kinds of useful and fun extensions for Krita. Like a new color picker and mixer, a plugin to modify the way Krita’s subwindows are handled, new toolboxes, integration with other applications like Autodesk Shotgun,

But what was missing was a good overview of the various areas that could be scripted. Tutorials and example code on how to use the scripting API in bite-size chunks. The regular API documentation is generated automatically from the APIDOX comments. It is a good reference but can be difficult to understand since it is generated from the C++ code that provides the scripting bindings.

Scott Petrovic, Krita’s UX designer and website maintainer, created a fresh new tutorial website: Scripting with Krita:

Scripting with Krita

And there are not just the lessons teaching you about creating scripts, running them, using the Krita scripting API and how to develop plugins, there are also really useful overviews of the actions your script can trigger and the icons you can use!

The post Announcing Krita’s Scripting School! appeared first on Krita.

Categories: FLOSS Project Planets

Specbee: Top Drupal 8 (and 9) Modules for Intuitive Website Navigation

Planet Drupal - Tue, 2020-08-04 07:55
Top Drupal 8 (and 9) Modules for Intuitive Website Navigation Shefali Shetty 04 Aug, 2020 Top 10 best practices for designing a perfect UX for your mobile app

What’s the secret sauce to a successful website? Well, there are more than 10 factors I can think of right off the bat. Like an attractive design, page-load speed, quality of content, marketing efforts and more.  One significant yet often overlooked element for a great user experience that drives a successful website is an intuitive navigation. Drupal 8 has a great set of modules to improve the navigation structure of your website. We have curated a list of top Drupal 8 (and 9) modules that enable easy and intuitive website navigation. Read on to find out.

A good website navigation lets site visitors know exactly where to get their information from as soon as they land on your website. Conversely, bad website navigation damages your rate of conversions and increases bounce rates. A website with attractive design does not always mean that navigating through it is intuitive. I have seen websites with ordinary designs with great navigation structures. And I keep going back to them because I know I can get what I need without scampering all over the place. According to CrazyEgg, the thumb rule is that it shouldn’t take more than 3 clicks for your user to find what they need.

Elements for an Intuitive Website Navigation

Your website visitors should be able to navigate from one page to another smoothly without getting distracted or confused. Distracted or confused users will leave your website before you establish a connection with them. Having a great design is good but if your visitors are not able to find your contact form, there is no point, is there? So, what makes up for a good navigation structure? 

1.    Main Navigation Bars 

This is the most crucial navigation element of a website. It is a horizontal (sometimes vertical) bar that lists links to point visitors around your website. A good main navigation bar needs to be simple, short, consistent, helpful, and catchy.

2.    Breadcrumbs

These helpful navigational aids help site visitors to identify where they exactly are. They are a trail of links that starts from the parent page and ends with the current page, usually separated with a “>” or a “/” symbol.

3.    Multi-column Menus (and Submenus)

A more complex website with tons of branches and sub-branches should use multi-column menus. Again, these menus should be simple and easy to navigate through. 

4.    Sitemaps

Often considered as just an SEO booster, Sitemaps are extremely helpful as a navigation aid as well. A typical Sitemap should display a hierarchical structure of the entire website.

5.    CTA (Call To Action) Buttons

This is where all the action actually happens! Proper placement of these CTA buttons play an important role amongst others like color, font style, size, text, etc.

6.    Sidebars

This is a good place to add page-specific links to enhance the UX for your site visitors. Good sidebars should be simple, not too long, contain a CTA and be ordered appropriately.

7.    Hyperlinks

Here we are talking more about internal linking than external. A hyperlink should be intuitive and lead the visitor to the page they expect (no surprises please). Although having internal links are good for SEO, don’t over-do it or you may risk losing their focus on the page.  

8.    Footers

Footers are easy to ignore and are often used only to display only copyright details. The myth that people don’t scroll till the end of a page is now busted. A typical footer should contain links that you are not able to display in the header or sidebar sections. It could also have your contact details as well as mailing list sign up mini forms.

Top Drupal 8 (and 9) Modules for Intuitive Navigation 1.    Menu Block 

The Drupal 8 Menu Block module’s design follows Drupal’s standard tree navigation style but  provides more enhanced features than the ones offered in the core Menu modules. You can easily configure blocks of menu links and specify the level you want to start and end with. There is no limit to the number of levels you can display. You can choose to keep your child menu elements expanded. Some basic features of this module are ported to Drupal 8 core. The Menu block module also supports Drupal 9.

2.    Easy Breadcrumb

The Drupal 8 Easy Breadcrumb module works by extracting location segments from the current URL of the webpage. It offers tons of configurable options and can replace the existing Drupal breadcrumbs module. You can choose to completely hide the home page link or any other specified links. The Easy breadcrumb module supports Drupal 9 too!

3.    Superfish

The Drupal 8 Superfish module is ideal when you have large and multiple layered, multi-column drop down menus. The module integrates with the jQuery Superfish plugin which is a highly versatile menu plugin that works for touchscreens, screen-readers and other keyboard interactions. It can be configured to add time delays on mouse-out, animations, hide and reveal menu links, adds arrows if submenus are detected, and much more. It is also Drupal 9 compatible.

4.    Sitemap

The Drupal 8 Sitemap module is simple, clean and easy to use. It can display the entire site structure in a clean hierarchical structure. You can also choose to generate and display RSS feeds for blogs and categories. The Sitemap module supports Drupal 9.

5.    Menu Item Role Access

This module allows you to add access control to your menus. You can enable and disable menu items depending on user roles. The Drupal 8 Menu Item Role Access module allows you to enter the role field (optional) to the menu items. It also supports Drupal 9.

          Image source: Menu Item Role Access
6.    Cheeseburger Menu

If you thought hamburger menus were great, wait till you check out the Cheeseburger menu. The Drupal 8 Cheeseburger menu module comes with some great features and is not just limited to mobile view, it also works with desktop views and varied screen sizes as well. It gives you the flexibility to choose the menu items that you want to appear in the Cheeseburger menu and also edit the menu titles. If you’re using Drupal Commerce, you can choose to display the shopping cart or the store phone number in the menu. Taxonomies could be chosen as the structure for your Cheeseburger menu. The Cheeseburger menu module supports Drupal 9 too!

Image source : Cheeseburger menu


7.    Simplify Menu

The Drupal 8 Simplify Menu module lets you render menus in your Twig template. It allows customization of the menu markup that enables accessibility and compatible with standards. You can render your menu tree as an array in the twig template and have full control over on the menu’s array. This module does not support Drupal 9 yet.

8.    Total Control Admin Dashboard

The Drupal 8 Total Control Admin Dashboard is a useful administrative navigation tool. It acts as a centralized hub for all administration tools. The dashboard displays admin panes and quick links to users, taxonomies, menus, content types, site stats, views panel panes and more. The views panel panes can be further customized. This module also works with Drupal 9.

9.    Footermap: a footer site map

As the name suggests, the Drupal 8 Footermap module provides a sitemap for your Drupal website that can be placed in the footer block. It allows dynamic generation of a sitemap and can be easily configured to support translation and caching. You can choose the menus to display, set the menu level limit (child menus), enable menu heading, edit the CSS to match your site layout and much more. The Footermap module also works with Drupal 9.


There are tons of benefits of having a good and intuitive website navigation system. When your visitors get the information they need without having to wander around for long, they will come back to you for more. Analytics tools and AB testing tools can be used to analyze and tweak the navigation elements of your website to help improve your existing navigation system. Drupal enables an engaging and intuitive navigation experience with its many core and contributed modules, some of which are listed above. As a leading Drupal development company, we strive to provide our customers with websites that are highly engaging and drives results. Connect with us today to see how we can help you with your next Drupal project.

Drupal Planet Drupal Module Drupal 8 Drupal Tutorial Shefali ShettyApr 05, 2017 Subscribe For Our Newsletter And Stay Updated Subscribe

Leave us a Comment

  Shefali ShettyApr 05, 2017 Recent Posts Image Top Drupal 8 (and 9) Modules for Intuitive Website Navigation Image Drupal Pathauto Module - A Brief Tutorial on how to Automatically Generate Bulk URL Aliases in Drupal 8 Image Headless Drupal – The All-You-Need-To-Know Guide Want to extract the maximum out of Drupal? TALK TO US Featured Success Stories

Know more about our technology driven approach to recreate the content management workflow for [24]7.ai


Find out how we transformed the digital image of world’s largest healthcare provider, an attribute that defined their global presence in the medical world.


Discover how a Drupal powered internal portal encouraged the sellers at Flipkart to obtain the latest insights with respect to a particular domain.

Categories: FLOSS Project Planets

Catalin George Festila: Python 3.6.9 : My colab tutorials - parts 008.

Planet Python - Tue, 2020-08-04 07:48
Today I deal with these two python packages named selenium and chromium-chromedriver. I used selenium to get pieces of information from webpages. These examples can be found at my GitHub project colab on the notebook named catafest_008.
Categories: FLOSS Project Planets

Lucius Digital: 23 cool Drupal modules for site builders and developers | August 2020

Planet Drupal - Tue, 2020-08-04 05:55
As a Drupal developer / consultant I think it is important to stay up-to-date on module releases, so that I don't code unnecessarily and manage projects as efficient as possible. So here's my batch of this month -all Drupal 9 compatible!
Categories: FLOSS Project Planets

Plasma Browser Integration 1.7.6

Planet KDE - Tue, 2020-08-04 03:00

I’m pleased to announce the immediate availability of Plasma Browser Integration version 1.7.6 on the Chrome Web Store as well as Firefox Add-Ons page. This release comes with a few bug fixes, performance improvements, and translation updates.

Konqi surfing the world wide web

Plasma Browser Integration bridges the gap between your browser and the Plasma desktop. It lets you share links, find browser tabs in KRunner, monitor download progress in the notification center, and control music and video playback anytime from within Plasma, or even from your phone using KDE Connect!

What’s new?

(also see the Changelog Page on our Community Wiki)

Detecting camouflaged Chromium-based browsers

Chromium-based browsers such as Vivaldi and Brave have recently changed their user agents to match that of regular Chrome to avoid being spuriously blocked. However, this also meant that media controls and other extension features were mapped to Chrome instead. This has been addressed by also checking our parent process – something a website can’t so, so you’ll still be impersonating a “proper” Chrome on the web. If there’s any other Chrome-based browser you want to see explicitly supported, let us know!

Better video iframe handling

The extension is loaded into every website as well as its iframes. When navigating away from the page that is currently playing, media controls will be rescinded. Due to a bug this did not work for iframes, leading to a popuplar bug where Plasma would show ghost videos playing on pages like Tweetdeck.

Moreover, the website’s DOM is also monitored to notice when a player is dynamically added or removed, for instance when a video overlay opens or closes. Unfortunately, this monitoring can be quite expensive and became a bottleneck on pages with lots of dynamically loaded content like is the case when viewing a large merge request on KDE’s GitLab instance. This release significantly improves performance here.

Improved media controls

As usual, this release brings some improvements to media controls. Short sounds and videos are currently ignored to avoid trying to control e.g. a “new message” sound or short hover video previews. However, some live stream implementations don’t report the correct duration of Infinity but gradually fill up their time bucket every few seconds. Previously, the extension only checked duration once to determine whether to provide media controls. With this update duration is continuously checked and media controls would become available eventually.

Furthermore, for websites that do not set album art through Media Session API, the video player’s poster is now used as album cover. This is the cover image that is often shown when the video isn’t playing.

Categories: FLOSS Project Planets

Django Weblog: Django 3.1 Released

Planet Python - Tue, 2020-08-04 01:22

The Django team is happy to announce the release of Django 3.1.

The release notes cover the potpourri of new features in detail, but a few highlights are:

You can get Django 3.1 from our downloads page or from the Python Package Index. The PGP key ID used for this release is Mariusz Felisiak: 2EF56372BA48CD1B.

With the release of Django 3.1, Django 3.0 has reached the end of mainstream support. The final minor bug fix release, 3.0.9, was issued yesterday. Django 3.0 will receive security and data loss fixes until April 2021. All users are encouraged to upgrade before then to continue receiving fixes for security issues.

See the downloads page for a table of supported versions and the future release schedule.

Categories: FLOSS Project Planets

Podcast.__init__: Supporting The Full Lifecycle Of Machine Learning Projects With Metaflow

Planet Python - Mon, 2020-08-03 23:14
Netflix uses machine learning to power every aspect of their business. To do this effectively they have had to build extensive expertise and tooling to support their engineers. In this episode Savin Goyal discusses the work that he and his team are doing on the open source machine learning operations platform Metaflow. He shares the inspiration for building an opinionated framework for the full lifecycle of machine learning projects, how it is implemented, and how they have designed it to be extensible to allow for easy adoption by users inside and outside of Netflix. This was a great conversation about the challenges of building machine learning projects and the work being done to make it more achievable.Summary

Netflix uses machine learning to power every aspect of their business. To do this effectively they have had to build extensive expertise and tooling to support their engineers. In this episode Savin Goyal discusses the work that he and his team are doing on the open source machine learning operations platform Metaflow. He shares the inspiration for building an opinionated framework for the full lifecycle of machine learning projects, how it is implemented, and how they have designed it to be extensible to allow for easy adoption by users inside and outside of Netflix. This was a great conversation about the challenges of building machine learning projects and the work being done to make it more achievable.

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • This portion of Python Podcast is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place (sometimes fast, sometimes slow)? Do you know why? With Datadog, you will. You can troubleshoot your app’s performance with Datadog’s end-to-end tracing and in one click correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at datadog.com/pythonpodcast. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
  • Your host as usual is Tobias Macey and today I’m interviewing Savin Goyal about Netflix’s infrastructure for machine learning
  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing the work you are doing at Netflix to support their machine learning workloads?
  • How are you addressing the impedance mismatch of machine learning/data science work between local experimentation and production deployment?
  • What was the motivation for building Metaflow?
    • How does Metaflow compare to other tools in the ecosystem such as MLFlow?
    • What was missing in the other available tools that made Metaflow necessary?
  • workflow for someone using Metaflow
  • How do you approach the design of the developer interface to make it approachable to machine learning engineers?
  • level of coupling with overall Netflix data stack
  • How is Metaflow implemented?
    • How has the architecture and design of the system evolved since you first began working on it?
  • supporting infrastructure/integration points
  • motivation/benefits of releasing it as open source
  • What are some of the most interesting, unexpected, or challenging lessons that you have learned while building infrastructure and tooling for machine learning?
  • When is Metaflow the wrong choice?
  • What do you have planned for the future of Metaflow and
Keep In Touch Picks Closing Announcements
  • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
  • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
  • If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
  • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Categories: FLOSS Project Planets

PSF GSoC students blogs: Week 9 Check-in

Planet Python - Mon, 2020-08-03 20:43
What did you do this week?

This week I continued the PR started in the previous week by adding more multimethods for indexing routines. The multimethods added are the following:

Generating index arrays

  • c_
  • r_
  • s_

Indexing-like operations

  • take
  • take_along_axis
  • choose
  • diagonal
  • select

Inserting data into arrays

  • place
  • put
  • put_along_axis
  • putmask
  • fill_diagonal

Iterating over arrays

  • nditer
  • ndenumerate
  • ndindex
  • nested_iters
  • lib.Arrayterator

Most of the work developed during this week was adding default implementations to the new mutimethods. Also, this is the first PR that makes use of the overriding of unumpy's classes to implement some classes for indexing routines. Unfortunately, I wasn't able to start the PR that adds multimethods for Numpy's random module that I set out to do in the last blog post. Nevertheless, since this is considered extra work there are no delays in the timeline. I will try to do this PR in the following weeks.

What is coming up next?

I will start a new PR that adds mutimethods for statistic functions. The work for this following week might be more demanding since the PR is expected to last only one week and it covers a considerable amount of multimethods. Also, some of the default implementations might add an extra challenge. Let's see how it goes! I also need to patch up the PR for the indexing routines which should be almost done.

Did you get stuck anywhere?

I didn't get stuck this week.

Categories: FLOSS Project Planets

Awesome Python Applications: buku

Planet Python - Mon, 2020-08-03 20:15

buku: Browser-independent bookmark manager with CLI and web server frontends, with integrations for browsers, cloud-based bookmark managers, and emacs.


Categories: FLOSS Project Planets

Agaric Collective: Double-edged Raiser: Past time to ditch Blackbaud

Planet Drupal - Mon, 2020-08-03 20:08

I have watched in sadness and sometimes anger as large non-profit after large non-profit collectively poured enough money into Raiser's Edge and other Blackbaud licenses and consulting services to fund many feature enhancements for the main FLOSS alternative, CiviCRM— improvements which would then be free for everyone, forever.

I have never met anyone who actually likes Blackbaud products and services. However, many organizations felt they were the only safe option, in the sense of claiming to have everything an enterprise needs.

Now, Blackbaud failed to secure its servers sufficiently and large amounts of its clients' donor data, including personally identifying information, was obtained in a ransomware attack. This was back in May. Blackbaud ultimately paid the ransomer to allegedly destroy the data they obtained— and only late in July finally told their customers what happened.

As the American Civil Liberties Union wrote to all its supporters, current and past (including myself), this is a rotten situation:

In all candor, we are frustrated with the lack of information we've received from Blackbaud about this incident thus far. The ACLU is doing everything in our power to ascertain the full nature of the breach, and we are actively investigating the nature of the data that was involved, details of the incident, and Blackbaud's remediation plans.

We are also exploring all options to ensure this does not happen again, including revisiting our relationship with Blackbaud.

Fortunately, none of Agaric's clients are affected. But we hope everyone using or considering using Blackbaud and other proprietary services for their most important data will look at free/libre open source solutions. Code you (or your technology partner) can see and contribute to means you truly can do anything. And if you put aside the money that would be gouged out of your organization by the eTapestry, Kintera, and Convio-swallowing monopolist Blackbaud, you probably can afford to.

At Agaric, we have recently been working with CiviCRM more recently (building on experience dating back fifteen years!) and we know our friends at Palante Technology Cooperative and myDropWizard are well-versed in CiviCRM, as are many others. Please consider this when weighing your options for maintaining a strong, ethical relationship with your supporters, and let us know if you have any thoughts or questions!

Read more and discuss at agaric.coop.

Categories: FLOSS Project Planets

Techiediaries - Django: Multiple File/Image Upload with Django 3, Angular 10 and FormData

Planet Python - Mon, 2020-08-03 20:00

In the previous tutorial we have seen how to implement file uploading in Django and Angular 10. In this tutorial, we'll see how to implement multiple file uploading with FormData.

It's recommended that you start from the previous tutorial to see detailed steps of how to create a django project, how to install Angular CLI and generate a new Angular 10 project along with services and components as we won't cover those basics in this part.

Working with Angular 10, Django and FormData

In this tutorial, we'll see how to use HTML5 FormData with Angular and Django to upload multiple files from a TypeScript frontend to a Python server.

Let's get started by cloning a django project with single file uploading already implemented.

Cloning Angular 10 Django Upload App

If you don't want to follow the steps from the previous part, you first need to get the project we've built. Open a new terminal and run the following command:

$ git clone https://github.com/techiediaries/django-angular-file-upload-example.git

Next, navigate inside the project's folder and install the npm dependencies using the following command:

$ cd django-angular-file-upload-example $ npm install

Next, start the development server using:

$ ng serve

Your Angular application will be available from the address.

Running the Django 3 Upload Server

Open a new terminal window and create a virtual environment using the following command:

$ python3 -m venv .env

Next, activate the virtual environment using:

$ source .env/bin/activate

Next, navigate to the backend project and install the Python packages using:

$ cd django-angular-file-upload-example/backend $ pip install -r requirements.txt

Finally, start the development server using:

$ python manage.py runserver

Open your web browser and navigate to the page where you can upload image files to the server:

Adding Multiple File Upload with Angular 10, TypeScript and FormData

Now, let's proceed to implement multiple file uploading.

As a reminder, before you can upload files in your django application, you need to set the MEDIA_URL and MEDIA_ROOT in your settings.py file:

MEDIA_URL = '/media/' MEDIA_ROOT = os.path.join(BASE_DIR, 'media') Installing ng2-file-upload

We will be using the ng2-file-upload library which provides easy to use directives for working with file upload in Angular 10:

$ npm install --save ng2-file-upload Importing the File Upload Angular Module

After installing this package, you will need to import FileUploadModule in your application module. Open the src/app/app.module.ts file and the following changes:

// [...] import { FileUploadModule } from 'ng2-file-upload'; @NgModule({ declarations: [ AppComponent, ProfileComponent ], imports: [ // [...] FileUploadModule ], providers: [], bootstrap: [AppComponent] }) export class AppModule { }

After adding FileUploadModule you'll be able to use the following directives in your templates:

  • The ng2FileDrop directive which will enable you to add an area where users can drag and drop multiple files,
  • The ng2FileSelect directive which will enable you to add an input button for selecting multiple files.
Adding the Upload Input

Open the src/app/profile/profile.component.html file and the following content:

<h1>Django REST API with Angular 10 File Upload Example</h1> <div ng2FileDrop [ngClass]="{'drop-file-over': hasBaseDropZoneOver}" (fileOver)="fileOverBase($event)" [uploader]="uploader" class="area"> <div id="dropZone">Drop files here</div> </div> <input type="file" ng2FileSelect [uploader]="uploader" multiple />

We add the ng2FileDrop directive to the <div> that represents the drop area and the ng2FileSelect directive to the file input field. We also add the multiple keyword to the file input to allow users to select multiple files.

We also use ngClass to add a dynamic CSS class to the drop area that gets activated when a file is dragged over the area and we bind it to the hasBaseDropZoneOver variable which will define in the component.

We bind the fileOver event to a fileOverBase() method that we'll also define in the component. This will be called when a file is dragged over the dropping area.

We also bind the uploader property to an uploader object that we'll also define in the component. This object is used to track the selected and dropped files that will be uploaded.

Next, we add a button to actually upload the files an a list to show the files that will be uploaded:

<button (click)="upload()">Upload files</button> <h2>Your files: {{ uploader?.queue?.length }}</h2> <ul> <li *ngFor="let item of uploader.queue"> {{ item?.file?.name }} </li> </ul>

Next, open the src/app/profile/profile.component.ts file and start by adding the following imports:

// [...] import { UploadService } from '../upload.service'; import { FileUploader, FileLikeObject } from 'ng2-file-upload'; import { concat } from 'rxjs';

Next, define the following variables:

DJANGO_SERVER = ''; public uploader: FileUploader = new FileUploader({}); public hasBaseDropZoneOver: boolean = false;

Next, define the fileOverBase() method which gets called when a file is dragged over the drop area:

fileOverBase(event): void { this.hasBaseDropZoneOver = event; }

The event variable equals true when the file is over the base area of the drop area.

Next, define the getFiles() method which return the array of files in the uploader queue:

getFiles(): FileLikeObject[] { return this.uploader.queue.map((fileItem) => { return fileItem.file; }); } Adding the TypeScript Upload Method

Finally, add the upload() method that will be called to actually upload the files to the Django server using HttpClient and FormData:

upload() { let files = this.getFiles(); console.log(files); let requests = []; files.forEach((file) => { let formData = new FormData(); formData.append('file' , file.rawFile, file.name); requests.push(this.uploadService.upload(formData)); }); concat(...requests).subscribe( (res) => { console.log(res); }, (err) => { console.log(err); } ); }

We call the getFiles() method to get an array of all the selected and dropped files. Next we loop over the files array and we create a FormData object and we append the current file in the loop to it then we call the upload() method of our UploadService and we push the returned Observable to the requests array.

This is another example of using FormData and TypeScript.

Finally we use the RxJS concat() operator to concatenate all returned Observables and subscribe to each one of them sequentially to send multiple POST requests to the server.

Note: In our example, we created a FormData object for each file in the files array. In theory we could create just one FormData object and append all the files in it using [] in the key i.e formData.append('file[]' , file.rawFile, file.name); then send only one request to the Django server to upload all the files appended to the FormData object (See FormData.append()) but this doesn't seem to work for us! (Maybe because of TypeScript?).

We'll use the CSS styling from this codepen. Open the src/app/profile/profile.component.css file and add:

.area { width: 77%; padding: 15px; margin: 15px; border: 1px solid #333; background: rgba(0,0,0,0.7); } #dropZone { border: 2px dashed #bbb; -webkit-border-radius: 5px; border-radius: 5px; padding: 50px; text-align: center; font: 21pt bold arial; color: #bbb; } .drop-file-over{ background: #333; }

This is a screenshot of the page after selecting and uploading a bunch of files:

Understanding FormData

Typically, when sending data through a form, it will be encoded with application/x-www-form-urlencoded encoding type. Except for when you need to use a file input field (i.e <input type="file">) in your form; in this case you need to use the multipart/form-dataencoding type.

The multipart/form-data can be used to send complex types of data such as files. Data is sent as key/value pairs where each value is associated with a key.

HTML5 provides the FormData interface which is equivalent to using a multipart/form-data form. This interface is useful when you want to send multipart form data with Ajax or HttpClient in case of Angular so instead of creating a form with the multipart/form-data type, we create an instance of FormData and we use the append() method to add key/value pairs.


In this tutorial, we've seen an example of multiple file upload with Angular 10 and Django 3.

Categories: FLOSS Project Planets

Holger Levsen: 20200803-debconf5

Planet Debian - Mon, 2020-08-03 18:13

This tshirt is 15 years old and from DebConf5. It still looks quite nice!

DebConf5 was my 3rd DebConf and took place in Helsinki, or rather Espoo, in Finland.

This was one of my most favorite DebConfs (though I basically loved them all) and I'm not really sure why, I guess it's because of the kind of community at the event. We stayed in some future dorms of the universtity, which were to be first used by some European athletics chamopionship and which we could use even before that, guests zero. Being in Finland there were of course saunas in the dorms, which we frequently used and greatly enjoyed. Still, one day we had to go on a trip to another sauna in the forest, because of course you cannot visit Finland and only see one sauna. Or at least, you should not.

Another aspect which increased community bonding was that we had to authenticate using 802.10 (IIRC, please correct me) which was an authentication standard mostly used for wireless but which also works for wired ethernet, except that not many had used it on Linux before. Thus quite some related bugs were fixed in the first days of DebCamp...

Then my powerpc ibook also decided to go bad, so I had to remove 30 screws to get the harddrive out and 30 screws back in, to not have 30 screws laying around for a week. Then I put the harddrive into a spare (x86) laptop and only used my /home partition and was very happy this worked nicely. And then, for travelling back, I had to unscrew and screw 30 times again. (I think my first attempt took 1.5h and the fourth only 45min or so Back home then I bought a laptop where one could remove the harddrive using one screw.

Oh, and then I was foolish during the DebConf5 preparations and said, that I could imagine setting up a team and doing video recordings, as previous DebConfs mostly didn't have recordings and the one that had, didn't have releases of them...

And so we did videos. And as we were mostly inexperienced we did them the hard way: during the day we recorded on tape and then when the talks were done, we used a postprocessing tool called 'cinelerra' and edited them. And because Eric Evans was on the team and because Eric worked every night almost all night, all nights, we managed to actually release them all when DebConf5 was over. I very well remember many many (23 or 42) Debian people cleaning the dorms thoroughly (as they were brand new..) and Eric just sitting somewhere, exhausted and watching the cleaners. And everybody was happy Eric was idling there, cause we knew why. In the aftermath of DebConf5 Ben Hutchings then wrote videolink (removed from sid in 2013) which we used to create video DVDs of our recordings based on a simple html file with links to the actual videos.

There were many more memorable events. The boat ride was great. A pirate flag appeared. One night people played guitar until very late (or rather early) close to the dorms, so at about 3 AM someone complained about it, not in person, but on the debian-devel mailinglist. And those drunk people playing guitar, replied immediatly on the mailinglist. And then someone from the guitar group gave a talk, at 9 AM, and the video is online... (It's a very slowwwwwww talk.)

If you haven't been to or close to the polar circles it's almost impossible to anticipate how life is in summer there. It get's a bit darker after midnight or rather after 1 AM and then at 3 AM it get's light again, so it's reaaaaaaally easy to miss the night once and it's absolutly not hard to miss the night for several nights in a row. And then I shared a room with 3 people who all snore quite loud...

There was more. I was lucky to witness the first (or second?) cheese and whine party which at that time took place in a dorm room with, dunno 10 people and maybe 15 kinds of cheese. And, of course, I met many wonderful people there, to mention a few I'll say Jesus, I mean mooch or data, Amaya and p2. And thanks to some bad luck which turned well, I also had my first time ever Sushi in Helsinki.

And and and. DebConfs are soooooooo good! I'll stop here as I originally planned to only write a paragraph or two about each and there are quite some to be written!

Oh, and as we all learned, there are probably no mosquitos in Helsinki, just in Espoo. And you can swim naked through a lake and catch a taxi on the other site, with no clothes and no money, no big deal. (And you might not believe it, but that wasn't me. I cannot swim that well.)

Categories: FLOSS Project Planets

Giovanni Mascellani: Bye bye Python 2!

Planet Debian - Mon, 2020-08-03 15:00

And so, today, while I was browsing updates for my Debian unstable laptop, I noticed that aptitude wouldn't automatically upgrade python2 and related packages (I don't know why, and at this point don't care). So I decided to dare: I removed the python2 package to see what the dependency solver would have proposed me. It turned out that there was basically nothing I couldn't live without.

So, bye bye Python 2. It was a long ride and I loved programming with you. But now it's the turn of your younger brother.

$ python bash: python: comando non trovato

(guess what "comando non trovato" means?)

And thanks to all those who made this possible!

Categories: FLOSS Project Planets

QML Online - Can be everywhere!

Planet KDE - Mon, 2020-08-03 14:59

A new feature of QML Online is already available, allows it to run in any site/blog with minimal js/html code!

Hopefully, our experience with QML examples, tutorials and documentation should change in the near future.

Ff you don't know what QML Online is, please take a look in my previous posts:

What are we talking about ?

QML Online now can be used in any blog or website without much work, like this:

import QtQuick 2.7 import QtQuick.Controls 2.3 import QtQuick.Layouts 1.12 Rectangle { color: "#179AF3" anchors.fill: parent ColumnLayout{ anchors.bottom: parent.bottom anchors.right: parent.right CheckBox { checked: true text: "Check this!" } CheckBox { text: "Or this!" } } Text { text: "KDE" font.pixelSize: 80 font.bold: true color: "#82CB38" anchors.centerIn: parent RotationAnimator on rotation { running: true loops: Animation.Infinite from: 0 to: 360 duration: 1500 } } } --> const qmlonline = new QmlOnline("qmlonline") qmlonline.registerCall({ qmlMessage: function(msg) { console.log(`qml message: ${msg}`) }, qmlError: function(data) { console.log(`qml message: ${JSON.stringify(msg)}`) }, posInit: function() { qmlonline.setCode(`import QtQuick 2.7 import QtQuick.Controls 2.3 import QtQuick.Layouts 1.12 Rectangle { color: "#179AF3" anchors.fill: parent ColumnLayout{ anchors.bottom: parent.bottom anchors.right: parent.right CheckBox { checked: true text: "Check this!" } CheckBox { text: "Or this!" } } Text { text: "KDE" font.pixelSize: 80 font.bold: true color: "#82CB38" anchors.centerIn: parent RotationAnimator on rotation { running: true loops: Animation.Infinite from: 0 to: 360 duration: 1500 } } }`) }, }) qmlonline.init()

And how can this new feature be used ?

It's quite simple, check this minimal HTML example:

<html> <head> <title>Qml Online minimal example</title> </head> <body> <script type="text/javascript" src="https://qmlonline.kde.org/qtloader.js"></script> <script type="text/javascript" src="https://qmlonline.kde.org/qml.js"></script> <div id="qmlonline"></div> <script type='text/javascript'> const qml = new QmlOnline("qmlonline") qml.registerCall({ qmlMessage: function(msg) { console.log(`qml message: ${msg}`) }, qmlError: function(data) { console.log(`qml error: ${JSON.stringify(data)}`) }, posInit: function() { qml.setCode(` import QtQuick 2.7 import QtQuick.Controls 2.3 Rectangle { color: "#179AF3" anchors.fill: parent Text { text: "KDE" font.pixelSize: 80 font.bold: true color: "#82CB38" anchors.centerIn: parent RotationAnimator on rotation { running: true loops: Animation.Infinite from: 0 to: 360 duration: 1500 } } } `) }, }) qml.init() </script> </body> </html>

As you can see, there is three steps, include both qtloader.js and qml.js, add a div DOM and create a QmlOnline object. Since I'm not a web expert, probably there is a better way to organize this approach for the user and bugs may exist. Be free to create Merge Requests, or get in touch with feature requests and issues.

What is next ?

From my planned objectives, sharing QML Online as library to be available for any website was one of the final points, the only one that's still missing is the Kirigami support, that's still in progress and hopefully will be finished until the end of the year (if everything goes fine).

There is also a small bug, where it's not possible to use multiple QML Online instances on the same webpage.

Categories: FLOSS Project Planets

PSF GSoC students blogs: Sorting Things Out

Planet Python - Mon, 2020-08-03 14:58

Hi! I really hope that everyone reading this is still doing okay, and if that isn't the case, I wish you a good day!

pip 20.2 Released!

Last Wednesday, pip 20.2 was released, delivering the 2020-resolver as well as many other improvements! I was lucky to be able to get the fast-deps feature to be included as part of the release. A brief description of this experimental feature as well as testing instruction can be found on Python Discuss.

The public exposure of the feature also remind me of some further optimization to make on the lazy wheel. Hopefully without download parallelization it would not be too slow to put off testing by concerned users of pip.

Preparation for Download Parallelization

As of this moment, we already have:

  • Multithreading pool fallback working
  • An opt-in to use lazy wheel to optain dependency information, and thus getting a list of wheels at the end of resolution ready to be downloaded together

What's left is only to interject a parallel download somewhere after the dependency resolution step. Still, this struggles me way more than I've ever imagined. I got so stuck that I had to give myself a day off in the middle of the week (and study some Rust), then I came up with something what was agreed upon as difficult to maintain.

Indeed, a large part of this is my fault, for not communicating the design thoroughly with pip's maintainers and not carefully noting stuff down during (verbal) discussions with my mentor. Thankfully Chris Hunt came to the rescue and did a refactoring that will make my future work much easier and cleaner.

Categories: FLOSS Project Planets