FLOSS Project Planets

Drupal Association Journey: Pedro Cambra: DrupalCon is coming up!

Planet Drupal - Sun, 2021-04-11 06:38

DrupalCon NorthAmerica starts tomorrow, but it is still not too late to register! There will be quite interesting content and networking activities and there’s a Drupal Association Q&A session where the floor is going to be open for the community to reach out the DA board and staff.

I’ve requested to be on the line up and I’ll be participating, so if you have any questions, please take your chance!

I don’t have a massive amount of updates from this last weeks regarding my board participation. I’ve been attending the Community and Finance committee meetings and there are some outcomes from there that I’ll be publishing about soon.

I’ve requested that the board minutes page is updated more often, and I’ll remind this to the person in charge of doing so.

I am preparing some proposals that hopefully will be addressed to the board regarding some considerations and process on the disenfranchisement non Drupal association in the elections issue. I hope to have updates on that too.

Note: This blog has the comments disabled, please feel free to send me a message through my contact page if you need to discuss anything related to the community and the Drupal Association. You can also tweet at me or find me in Drupal Slack or the distributed matrix network as pcambra.

#Drupal

Categories: FLOSS Project Planets

hussainweb.me: Thriving on Chaos in Drupal

Planet Drupal - Sat, 2021-04-10 23:46
Today, I want to share my thoughts from a book passage related to Drupal. The book, Everyday Chaos by David Weinberger, is largely about how chaos is the new reality in today's machine-learning-driven world. In this book, Drupal is discussed in the chapter on strategy and possibility where it is contrasted with more traditional methods of product development and organizational vision. The book is amazing and insightful, and the section on Drupal was a welcome surprise.
Categories: FLOSS Project Planets

Junichi Uekawa: Wrote a timezone checker page.

Planet Debian - Sat, 2021-04-10 22:06
Wrote a timezone checker page. timezone. Shows the current time in blue line. I haven't made anything configurable but will think about it later.

Categories: FLOSS Project Planets

Charles Plessy: Debian Bullseye: more open

Planet Debian - Sat, 2021-04-10 18:21

Debian Bullseye will provide the command /usr/bin/open for your greatest comfort at the command line. On a system with a graphical desktop environment, the command should have a similar result as when opening a document from a mouse-and-click file browser.

Technically, /usr/bin/open is a symbolic link managed by update-alternatives to point towards xdg-open if available and otherwise run-mailcap.

Categories: FLOSS Project Planets

Kentaro Hayashi: Grow your ideas for Debian Project

Planet Debian - Sat, 2021-04-10 09:13

There may be some "If it could be ..." ideas for Debian Project. If idea is concreate and worth to make things forward, it should make a proposal for Project Funding.

salsa.debian.org

But it is a just an idea, or no afford to act as an executor role, that idea will not be achieved.

I thought that It needs an incubator - complemental project.

salsa.debian.org

I've salvaged an idea from closed MR Add proposal about "Formalize reimbursement process" (!5) · Merge Requests · Freexian SARL / Project Funding · GitLab

I'm not confident whether mechanism works, but Debian needs change.

Categories: FLOSS Project Planets

Google does not want you to tell your players about your donation page

Planet KDE - Sat, 2021-04-10 08:48

I recently updated Pixel Wheels banner image on Google Play. That triggered a review of the game: shortly after the update I received a message telling me Pixel Wheels was "not compliant with Google Play Policies". What nefarious activity does the game engage in? Sneak on users? Mine bitcoins?

Nope.

Something much more terrible, as evidenced by this screenshot they sent me:

The Horror

Yes. I confess it. I added a link to my donation page within the game, depriving Google of some precious money it totally cannot survive without! How dare I?!? I am such a bad person.

Since there is no point arguing with them, I am going to have to build a Google Play flavor of the game, where this link is replaced with a link to the game page. Hopefully the Great Algorithm will accept that. We'll see.

Meanwhile you can still get the game from F-Droid or itch.io, since they do not have a problem with a link to a donation page.

Categories: FLOSS Project Planets

This week in KDE: Activities on Wayland

Planet KDE - Sat, 2021-04-10 00:17

This week the Wayland train continued barreling on, full speed ahead! We picked up a bunch of nice fixes and a big feature:

New Features

The “Activities” feature now mostly works on Wayland! There are a few remaining things to implement to make it 100% comparable to the X11 version, but that should get done in time for the next Major Plasma release (Kevin Ottens, Plasma 5.22)

Sticky Note widgets now have an option to change the font size (Shantanu Tushar, Plasma 5.22):

Bugfixes & Performance Improvements

Zooming in and out in Okular now works correctly when using the “Trim Margins” feature (Gerd Wachsmuth, Okular 21.04)

Media9 PDF movie annotations can once again be played in Okular (Albert Astals Cid, Okular 21.04)

When using Okular’s “Invert Luma/Lightness” setting, the loading page now retains its correct color (David Hurka, Okular 21.04)

Ark can now un-archive zip files with Windows-style backslashes used as path separators (João Silva, Ark 21.08)

Fixed a bug in the Breeze application style that could manifest as a big ugly black square appearing in KMail (Fabian Vogt, Plasma 5.18.8)

Fixed one way that Plasma could crash right after login (John Zimmermann, Plasma 5.21.4)

The Plasma Wayland session will no longer crash if you plug in an external screen while in a non-GUI session (e.g. a virtual terminal) (Jan Blackquill, Plasma 5.21.4)

The Bluetooth applet’s tooltip no longer displays the wrong name of the currently connected device. I originally fixed this 9 months ago in Plasma 5.19.1 but somehow the fix was never merged into Plasma 5.20, so it got broken again. That has now been corrected (me: Nate Graham with help from James John, Plasma 5.21.4)

Ultra-wide screens with a 21:9 aspect ratio are now displayed as “21:9” in System Settings’ Display Configuration page, rather than “63:27” (lol) (Felipe Kinoshita, Plasma 5.21.4)

Fixed one way that KWin could crash with certain low-power embedded GPUs (Vlad Zahorodnii, Plasma 5.21.5)

Maximized GTK app windows are no longer positioned too high in the Plasma Wayland session (Vlad Zahorodnii, Plasma 5.21.5)

Discover’s ability to show you an app’s dependencies now works again (Aleix Pol Gonzalez, Plasma 5.21.5)

Disconnecting a screen in the Plasma Wayland session no longer causes all Qt apps to crash (Vlad Zahorodnii, Plasma 5.22)

Global shortcuts are now working even on non-US keyboard layouts (Andrey Butirsky, Plasma 5.22 in conjunction with a Qt version that has this pending patch integrated)

Plasma no longer lags or hangs when displaying a massive number of tooltips for grouped Task Manager tasks (Aleksei Nikiforov, Plasma 5.22)

The kglobalaccel5 daemon can no longer block re-login by crashing on the previous log-out and then getting stuck (David Edmundson, Plasma 5.22 or Frameworks 5.82; whichever one you get first)

When using a multi-screen setup, the lock screen no longer only displays typed text on the text field of the left-most screen, even if you clicked on the text field on a different screen (Aleix Pol Gonzalez, Plasma 5.22)

The Task Manager’s “Highlight windows when hovering over tasks” feature now works in the Plasma Wayland session (David Redondo, Plasma 5.22)

Kate and other KTextEditor-based apps no longer crash if you delete an open file on disk and choose the “Close file, discarding contents” option in the warning message that appears in the app (Christoph Cullmann, Frameworks 5.82)

Fixed a rare case where Kate and other KTextEditor-based apps could crash when dragging text (Waqar Ahmed, Frameworks 5.82)

Context Menus for text fields inside Kirigami overlay sheets are no longer displayed below the sheet content (Noah Davis, Frameworks 5.82)

In Kate and other KTextEditor-based apps, the code completion pop-up no longer sometimes take up the whole screen width (Waqar Ahmed, Frameworks 5.82)

Text in Plasma tab buttons (such as in the new Kickoff menu) now gets elided when there’s not enough space, rather than overflowing (David Edmundson, Frameworks 5.82)

User Interface Improvements

Konsole’s “Edit Profile” window now displays errors inline, rather than using an ugly modal dialog window (Ahmad Samir, Konsole 21.04):

Okular’s “Continuous” mode is now considered to be a document-specific setting (like the zoom settings are), rather than a global setting (Mahmoud Khalil, Okular 21.08)

Items in the System Settings KWin Scripts now use the “pending deletion” pattern used in many other pages, whereby deleting an item only marks as it as in a “pending deletion” state and it only actually gets deleted when you click the “Apply” button (Alexander Lohnau, Plasma 5.22)

System Tray applets now receive keyboard focus when opened, so they can be interacted with using the keyboard (Eugene Popov, Plasma 5.22)

Hover buttons in the Clipboard System Tray applet’s list items are now top-aligned for tall ones, so that the trash button doesn’t shift around according to height, which makes it easy to click on that button repeatedly to manually prune your history list (me: Nate Graham, Plasma 5.22):

https://i.imgur.com/NjZKUY0.mp4

Discover no longer shows a huge weird rapidly-disappearing tooltip while loading the Updates page if the cursor is over any part of it (me: Nate Graham, Plasma 5.22)

The tooltip for the window decoration button used to keep a window above all others now makes its purpose more clear (me: Nate Graham, Plasma 5.22):

The previously somewhat confusing “Keyboard Indicator” applet has been renamed and given a UI overhaul to clarify what it is and what it does (Andrey Butirsky and me: Nate Graham, Plasma 5.22):

https://i.imgur.com/474lJL0.mp4

The Task Manager tooltip now visually indicates when it’s scrollable by displaying a visible scrollbar (me: Nate Graham, Plasma 5.22)

When using the systemwide double-click mode, it’s now possible to disable “click a selected file’s label to rename it” feature for desktop icons, just as it is in Dolphin (me: Nate Graham, Plasma 5.22)

Discover’s view of an app’s dependencies has received a visual overhaul and now also shows you the exact name of the package for the app in question and also groups the dependencies by installation status (Aleix Pol Gonzalez and me: Nate Graham, Plasma 5.22):

Looks like this will install all of GNOME; guess I don’t wanna do that

All the grid view pages in System Settings now sort items case-insensitively (me: Nate Graham and Alexander Lohnau, Plasma 5.22)

Plasma list items now have left and right margins that are consistent with their top and bottom margins (Noah Davis, Frameworks 5.82)

Various message dialogs throughout KDE software no longer display pointless tooltips saying, “Yes” and “No” when you hover your cursor over buttons whose own text may already be “Yes” and “No”! (me: Nate Graham, Frameworks 5.82)

Web Presence

Check out Niccolò’s video about how to set up a development environment and submit a merge request. Very handy for audiovisual learners!

…Read that as “dev env” not “ded end” lol …And everything else

Keep in mind that this blog only covers the tip of the iceberg! Tons of KDE apps whose development I don’t have time to follow aren’t represented here, and I also don’t mention backend refactoring, improved test coverage, and other changes that are generally not user-facing. If you’re hungry for more, check out https://planet.kde.org/, where you can find blog posts by other KDE contributors detailing the work they’re doing.

How You Can Help

Have a look at https://community.kde.org/Get_Involved to discover ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!

Finally, consider making a tax-deductible donation to the KDE e.V. foundation.

Categories: FLOSS Project Planets

hussainweb.me: Drupal Podcasts (and some PHP ones)

Planet Drupal - Fri, 2021-04-09 23:47
Today's DrupalFest post is on the lighter side. I am just going to talk about some of the podcasts I listen to related to Drupal, PHP, and software development in general. I'll try to cover all the Drupal podcasts I know about. Let me know in the comments if I have missed something. As for others, I am just listing those I listen to.
Categories: FLOSS Project Planets

Matt Glaman: Watch: Nightwatch testing training to help get Olivero stable for Drupal 9.2!

Planet Drupal - Fri, 2021-04-09 22:39

I am a little late getting this blog published. The end of March and early part of April just blew right past. However, I wanted to share the recordings from the Nightwatch.js training I gave at MidCamp on March 25th. I showed folks how to run Drupal's Nightwatch.js test suite locally, with DDEV and Lando.

The workshop code is available on the Bluehorn Digital GitHub organization, with documentation. I plan to develop this for all future testing workshops and keep it open for folks to learn from outside of paid workshops. In fact, the repo runs GitHub Actions to test the various setups and executes Drupal tests – the tests run tests to test that the tests can be tested! 😵

Categories: FLOSS Project Planets

KDE Ships Frameworks 5.81.0

Planet KDE - Fri, 2021-04-09 20:00

Saturday, 10 April 2021

KDE today announces the release of KDE Frameworks 5.81.0.

KDE Frameworks are 83 addon libraries to Qt which provide a wide variety of commonly needed functionality in mature, peer reviewed and well tested libraries with friendly licensing terms. For an introduction see the KDE Frameworks release announcement.

This release is part of a series of planned monthly releases making improvements available to developers in a quick and predictable manner.

New in this version Baloo
  • [SearchStore] Explicitly narrow timestamps for range query
  • Add now mandatory args parameter to QProcess::startDetached()
  • [MetadataMover] Update filename terms when moving/renaming file (bug 433116)
BluezQt
  • Fix unity compile support
Breeze Icons
  • Added branches with leaves to Kmymoney icon
  • Add a few symlinks for “configure” and “help-donate” (bug 435150)
  • Add KMyMoney Pie-Chart Icon
  • Link svn-* icons to new vcs-* icons
  • Add vcs-* icons for Kate
  • Make lock icon filled status consistent (bug 244542)
  • Remove 22 brightness icons in 16 size folder
  • Fix location of brightness icons
  • Add high-brightness and low-brightness icons
Extra CMake Modules
  • ECMGenerateExportHeader: do sanity check for version argument values
  • Fix warning about wayland-scanner code arg
KActivities
  • Activate activity manager asynchronously
KAuth
  • Un-overload HelperProxy::progressStep() signal
KCMUtils
  • Add loaded signal to KCModuleData to handle delayed loading
KCompletion
  • Un-overload KLineEdit::returnPressed(const QString &) signal => returnKeyPressed
  • Un-overload KCompletionBox::activated(const QString &) signal => textActivated
  • Un-overload KComboBox::returnPressed(const QString &) signal, by deprecating returnPressed()
KConfig
  • Relicense file to LGPL-2.0-or-later
  • kconfig_compiler: Explicitly open input file for reading
  • kconfig_compiler: change how paramString() creates strings
KConfigWidgets
  • Introduce KHamburgermenu (bug 368421)
KCoreAddons
  • Enable Unicode support in QRegularExpression where needed
KCrash
  • Document why we close FDs
KDED
  • Make kded shut down cleanly during systemd session teardown
KDELibs 4 Support
  • KComponentData: add a link to the KF5 porting notes
KFileMetaData
  • Enable Unicode support in QRegularExpression where needed
KGlobalAccel
  • Only use unistd/getuid when available
  • Don’t let kglobalaccel run if KDE_SESSION_UID mismatches
KHolidays
  • Make it compile with unity cmake support
  • Handle negative years in easter and pascha calculations (bug 434027)
KIconThemes
  • Revert “avoid race condition on loading the plugin”
  • Revert “add private header to avoid extern in .cpp file”
  • Don’t register our engine per default
  • More robust handling of missing global KDE themes
  • Ensure qrc + QDir::searchPaths work for icons (bug 434451)
  • More robust and complete setup of the scaled test environment
  • Produce output with the request devicePixelRatio
  • Properly render non-square icons
  • Remove the assumption that SVG icons are squares in icon loading
  • Retain non-square icon sizes in KIconEngine::actualSize()
  • Revert icon size behavior changes
  • Align handling of non-square icons with how Qt behaves
KIO
  • FileCopyJob: fix regression when copying a data: URL
  • Handle errors during xattr copy in a more robust way
  • Port ktelnetservice away from kdeinitexec
  • Handle .theme files correctly (bug 435176)
  • Remove KCoreDirListerCache::validUrl, let the job emit error instead
  • PreviewJob: Initialize cachesSize with 0, only pass size > 0 to shmget, improve createThumbnail (bug 430862)
  • KNewFileMenu: use destination side to stat destination (bug 429541)
  • MimeTypeFinderJob: don’t put job on hold for local files
  • FileCopyJob: port to the async AskUserActionInterface
  • Fix crash in ApplicationLauncherJob(service) when service is null
  • Don’t try to get mimetypes on empty urls
  • Fix appending file extensions in KFileWidget
Kirigami
  • Add a humanMoment unit to Kirigami.Units
  • Make the luma conversion of a color api accessible
  • Auto fire SearchField’s accepted, with optional extra delay (bug 435084)
  • Make globaltoolbar colorset customizable
  • Lower duration to change color for ActionButton
  • Fix focus handling in OverlaySheet to be managed as one FocusScope (bug 431295)
  • BasicListItem: partially silence binding loop
  • [FormLayout] Use layout boundaries on twin layout hints (bug 434383)
  • Make SwipeNavigator only allow the active page to be focused
  • Consider Ubuntu Touch to be mobile
  • [controls/Avatar]: Get rid of ‘both point size and pixel size’ set warning
  • Remove link to deprecated ApplicationHeader
  • the visible part should always at least be as tall as item (bug 433815)
  • Fix potential crash in SizeGroup (bug 434079)
  • turn contentItemParent into a FocusScope (bug 433991)
KJobWidgets
  • Introduce KUiServerV2JobTracker
KNewStuff
  • qtquickengine: Do not forward intermediate states
  • quickengine: Emit entryEvent signal with enum which is exposed to QML
  • Create a NewStuff.Action component, add NewStuff.Settings global
  • Less risk of infinite spinner on uninstalling KPackage based things (bug 434371)
KNotification
  • Relicense files to LGPL-2.0-or-later
  • Don’t close resident notifications when action is invoked
  • Implement inline replies on Android
  • Add an inline reply notification to the example
  • Add KNotificationReplyAction for using inline-reply Notification API
KParts
  • Add a new signal to replace the now deprecated completed(bool)
KRunner
  • Deprecate concept of delayed runners & related methods
  • Deprecate methods to remove matches in RunnerContext
KService
  • Deprecate KPluginInfo::fromKPartsInstanceName, completely unused
KTextEditor
  • Don’t warn about unsaved changes when closing if blank and unsaved (bug 391208)
  • Use QPalette::highlight for the scrollbar minimap slider (bug 434690)
  • Avoid gaps in indentation line drawing
  • Don’t use F9 & F10 as shortcuts
  • Use Okular’s QScroller settings
  • Add basic touchscreen support
  • [Vimode] Improve sentence text object
  • [Vimode] Fix paragraph text object in visual mode
  • Restrict horizontal range of cursor to avoid unintentionally wrapping (bug 423253)
  • Turn on line numbers & modification markers
  • Update remove-trailing-spaces modeline
  • Add option to keep spaces to the left of cursor when saving (bug 433455)
  • Remove unneeded options setting
  • Search: Enable Unicode support in QRegularExpression
  • [Vimode] Show search wrapepd hint for # and * motions
  • [Vimode] Use ViewPrivate::showSearchWrappedHint() for consistency;
  • Move showSearchWrappedHint() to KTextEditor::ViewPrivate
  • [Vimode] Only display “Search wrapped” message for n/N motions
  • Ensure we use unicode regex where needed
  • Fix spellcheck word detection for non-ASCII (bug 433673)
  • Fix auto-completion for non ASCII words (bug 433672)
KTextWidgets
  • Deprecate the KFind::highlight(int, int, int) signal
  • Deprecate the KFind::highlight(QString &, int, int) signal
  • Enable Unicode support in QRegularExpression where needed
KWallet Framework
  • Un-overload OrgKdeKWalletInterface::walletClosed(int) signal
KWayland
  • Bump required PlasmaWaylandProtocols
  • Fix DTD check errors and a typo
  • Add the activity management protocol client implementation
KWindowSystem
  • Add MediaPause key to mapping (bug 403636)
Plasma Framework
  • Deprecate AppletScript::description()
  • [widgets/arrows] Fix viewBox size and naturalSize in SvgItems
  • Add a humanMoment unit to the various Units
  • Add PageIndicator
  • [pluginloader] Add methods to list containments using KPluginMetaData
  • Deprecate PluginLoader::listEngineInfoByCategory
  • Mark Plasma Style as internal
  • Add notes about classes that will hopefully be dropped in KF6
  • [lineedit.svg] Remove empty space around borders and use 3px corner radius
  • [PC3 TextField] set on placeholder text
  • [PC3 TextField] mirrored -> control.mirrored
  • [PC3] Refactor TextField
  • Make compositing-off margins in the panel same size of compositing on
  • widgets>lineedit.svg: remove double ring
  • Change ContrastEffect check to AdaptiveTransparency in A.T. check (bug 434200)
  • Revert recent theme changes (bug 434202)
  • Port to singleton Units
QQC2StyleBridge
  • Respect highlighted property (bug 384989)
  • Fix size of toolbuttons
  • [DialogButtonBox] Improve implicit size behavior
Syntax Highlighting
  • LaTeX: allow Math env within another Math env
  • java: add assert and var keywords
  • cmake.xml: Updates for CMake 3.20
  • PHP: add preg_last_error_msg() function and classes that was previously a resource
  • Js: numeric separator ; fix Octal ; add new classes
  • Try to fix Qt 5.14 based compile
  • Make it possible to compile with Qt 6
  • Fix #5 and add Path style with alternate value (${xx:-/path/})
  • Python: add match and case keywords (Python 3.10)
  • Bump rust.xml to version 12
  • Do not spellcheck in Rust code
  • Markdown: add folding in sections
  • Fix BracketMatching color in Breeze Dark theme
  • Added more checking and number support
  • Added proper escaping support with error handling
  • Yet another DetectSpaces at MultiLineText
  • Removed Variable context; not needed
  • Added Error displaying for Placeables
  • Included ##Comments
  • Optimisations (DetectIdentifier, etc.)
  • Fixed Attribute handling
  • Added support for identifiers
  • Multiline text is now working
  • Added support for fluent language files
  • Add findloc Fortran function
ThreadWeaver
  • exception.h: fix export header include to work in namespace-prefixed include
Security information

The released code has been GPG-signed using the following key: pub rsa2048/58D0EE648A48B3BB 2016-09-05 David Faure faure@kde.org Primary key fingerprint: 53E6 B47B 45CE A3E0 D5B7 4577 58D0 EE64 8A48 B3BB

Categories: FLOSS Project Planets

Wingware: Wing Python IDE 7.2.9 - April 12, 2021

Planet Python - Fri, 2021-04-09 17:45

Wing 7.2.9 adds remote development for 64-bit Raspberry Pi, improves auto-closing of quotes, optimizes change tracking when large numbers of project files change at once, improves debugger data display for some value types, and makes a number of other usability improvements.

See the change log for details.

Download Wing 7.2.9 Now: Wing Pro | Wing Personal | Wing 101 | Compare Products


What's New in Wing 7.2

Auto-Reformatting with Black and YAPF (Wing Pro)

Wing 7.2 adds support for Black and YAPF for code reformatting, in addition to the previously available built-in autopep8 reformatting. To use Black or YAPF, they must first be installed into your Python with pip, conda, or other package manager. Reformatting options are available from the Source > Reformatting menu group, and automatic reformatting may be configured in the Editor > Auto-reformatting preferences group.

See Auto-Reformatting for details.

Improved Support for Virtualenv

Wing 7.2 improves support for virtualenv by allowing the command that activates the environment to be entered in the Python Executable in Project Properties, Launch Configurations, and when creating new projects. The New Project dialog now also includes the option to create a new virtualenv along with the new project, optionally specifying packages to install.

See Using Wing with Virtualenv for details.

Support for Anaconda Environments

Similarly, Wing 7.2 adds support for Anaconda environments, so the conda activate command can be entered when configuring the Python Executable and the New Project dialog supports using an existing Anaconda environment or creating a new one along with the project.

See Using Wing with Anaconda for details.

And More

Wing 7.2 also introduces support for Python 3.9, adds How-Tos for using Wing with AWS and PyXLL, makes it easier to debug modules with python -m, adds support for Python 3 enums, supports remote development to 64-bit Raspberry Pi, simplifies manual configuration of remote debugging, allows using a command line for the configured Python Executable, supports constraining Find Uses of imported symbols to only the current file, improves vi mode, allows folding .pyi and .pi files, improves Debug I/O process management, enhances accuracy of some types of code warnings, supports remote development without SSH tunnels, improves debugger data display for some value types, improves support for recent macOS versions, and makes a number of usability and stability improvements.

For a complete list of new features in Wing 7, see What's New in Wing 7.


Try Wing 7.2 Now!

Wing 7.2 is an exciting new step for Wingware's Python IDE product line. Find out how Wing 7.2 can turbocharge your Python development by trying it today.

Downloads: Wing Pro | Wing Personal | Wing 101 | Compare Products

See Upgrading for details on upgrading from Wing 6 and earlier, and Migrating from Older Versions for a list of compatibility notes.

Categories: FLOSS Project Planets

TestDriven.io: Django vs. Flask in 2021: Which Framework to Choose

Planet Python - Fri, 2021-04-09 17:45
In this article, we'll look at the best use cases for Django and Flask along with what makes them unique, from an educational and development standpoint.
Categories: FLOSS Project Planets

Community Working Group posts: 2020 Year-in-review for the Drupal Community Working Group

Planet Drupal - Fri, 2021-04-09 11:58

The past year has been a busy one for the Drupal Community Working Group (CWG), as we created a new "Community Health Team" and saw the stepping down of the last original member of the Conflict Resolution Team, George DeMet. As the CWG enters its 8th year, we feel it is our duty to continue to pursue our mission to "foster a friendly and welcoming community for the Drupal project and to uphold the Drupal Code of Conduct". With this guiding principle, we have been focusing on both proactive and reactive tasks to help us achieve this goal. 

This annual report will serve as a summary of what we've accomplished over the past year, as well as a discussion of some of our goals for the near future.

Community Health Team

The Community Working Group was expanded during the first half of 2020 with the creation of the Community Health Team. The mission of this new team is to focus on proactive community health tasks including workshops and knowledge transfer. With the help of Tara King, the CWG membership coordinator, we structured the team into several groups. Although team members may do work across multiple groups, each of these groups is designed to, but not limited to, focus on a specific area:

  • Community Event Support - provide resources and support related to the Code of Conduct for Drupal events.
  • Community Health - provide opportunities to educate and train community members to be more effective contributors.
  • Membership - to help identify and recruit community members for the CWG. 
  • Ambassadors - provide expertise and advice related to geographic, cultural, and other differences both inside and outside the Drupal community.

Community Health Team members are not privy to Code of Conduct incident reports; however they must adhere to the CWG Code of Ethics.

Once the team was created and volunteers were found for the majority of the roles, we began having monthly meetings during the second half of 2020. The team has already completed a number of tasks including:

  • Initial work on a Drupal Code of Conduct update.
  • Documentation of CWG roles.
  • Development of a group of community health representatives from other open source communities.
  • Ongoing Code of Conduct contact workshops.
  • Updates to the Drupal event Code of Conduct templates and playbook.
  • Ongoing Mental Health First Aid workshops for community members.
  • Blog posts related to community health.
  • "Nudges" for Drupal Slack Workspace and issue queues.

Other, long term goals for the Community Health Team include providing an on-ramp for the Conflict Resolution Team and identifying and presenting additional community-health-related workshops for the community, 

Conflict Resolution Team

After six years on the Conflict Resolution team, including several years as its chair, George DeMet retired from the team at the end of 2020. We cannot understate how much of an impact George has had on the CWG and the Drupal community, often working behind the scenes. We are fortunate that George has agreed to stay on as a member of the Community Health Team where he will be focusing on updating the Drupal Code of Conduct. 

During 2020, in addition to the creation of the Community Health Team, the Conflict Resolution Team continued to work on on-going and new Code of Conduct related issues. During our weekly meetings, we generally work on three types of tasks:

  • Internal business - examples include recruitment, public blog posts and presentations, Aaron Winborn Award, event organizer requests.
  • External, old business - ongoing conflict resolution tasks normally brought to us from community members.
  • External, new business - new conflict resolution tasks, normally brought to us from community members.

While some conflict resolution tasks can be resolved quickly (a few days), we normally have several long-term, on-going issues that can take anywhere from weeks to months to resolve. Most of the long-term issues include ongoing personality conflicts within the community, but we also routinely work with community members who had previously had their community privileges limited on plans and tasks to have those privileges restored (see our Balancing Accountability and Compassion in the Drupal Community blog post).

What types of conflict resolution issues do we work on?

We decided to perform a quantitative analysis of the number and types of conflict resolution issues we work on, comparing data from 2019 with 2020. Our methodology allowed us to assign one or two of the following categories to each new issue we received during 2019 and 2020:

  • Social media conflict
  • Issue queue conflict
  • Drupal Slack workspace conflict
  • In-person Drupal event conflict
  • Virtual Drupal event conflict
  • Not CWG domain
  • Other - examples include content issues on Drupal.org, issues related to local Drupal communities (but not directly related to an event), interpersonal issues occurring in areas not covered by any of the other categories.

In terms of overall number of incidents, while 2019 had 35 total new reported incidents to the CWG, 2020 has slightly less than half of that, with 17 new reported incidents. 

  • While the number of incidents occurring at in-person Drupal events dropped from six in 2019 to none in 2020, this doesn't account for the entire reduction of total incidents between 2019 and 2020. We also saw fewer social media and Drupal Slack workspace conflicts, but the biggest drop was in the "Other" category, which saw a decrease from ten incidents in 2019 to just two in 2020. 
  • Obviously, the drop in in-person incident reports is directly related to the pandemic.
  • What else can we attribute the dramatic drop in incident reports to? We hope that the formation of the Community Health Team is having some effect, but we're not so naive to attribute the entire decrease to its creation and actions during 2020. 

2019

2020

Total number of new reported issues

35

17

Social media conflict

7

1

Issue queue conflict

9

9

Drupal Slack workspace conflict

5

1

In-person Drupal event conflict

6

0

Virtual Drupal event conflict

0

2

Not CWG domain

4

3

Other

10

2

Looking forward Conflict resolution team membership

One of the primary goals of the conflict resolution team in the first part of 2020 was expanding the size of the team. With the recent departure of George DeMet and the decrease in our workload (thanks to fewer incident reports and the amazing work of the Community Health Team), we decided this was a good time to recruit new team members. 

We had six amazing community members approach us about joining the team, and will be inviting a new member(s) to the team in the next few weeks. One of the main goals of the Community Health team was to provide an on-ramp to the Conflict Resolution Team. Those community members who were not extended an offer to join the Conflict Resolution Team will be asked to join the Community Health Team in a capacity of their choosing, if they haven’t joined already.

As part of the process of having new members join the team, we implemented (and are in the process of documenting) a new on-boarding process, where new team members are considered "trial members" for a maximum of 5 months. During this period, new members will mainly shadow the team and have limited access to historical conflict resolution reports. At the conclusion of the trial period new members will either become regular members or be asked to leave the team. As is prescribed by our charter, all trial members must be approved by the CWG Review Panel

Community Health Team

Now that our Community Health Team is a year old and has some experience under its belt, we have high hopes that they will continue to be a force for good in the community. Our plans for the next year include finding and presenting additional workshops, completing the aforementioned Drupal Code of Conduct update, and assisting with the expansion of the yet-to-have-a-good-name group of community health volunteers from various open source communities.

Categories: FLOSS Project Planets

Learn PyQt: PyQt6 Book now available: Create GUI Applications with Python & Qt6 — The hands-on guide to making apps with Python

Planet Python - Fri, 2021-04-09 11:00

Hello! Today I have released the first PyQt6 edition of my book Create GUI Applications, with Python & Qt6.

This update follows the 4th Edition of the PyQt5 book updating all the code examples and adding additional PyQt6-specific detail. The book contains 600+ pages and 200+ complete code examples taking you from the basics of creating PyQt applications to fully functional apps.

To celebrate the milestone, the book is available this week with 20% off. As with earlier editions, readers get access to all future updates for free -- so it's a great time to snap it up! You'll also get a copy of the PyQt5, PySide6 and PySide2 editions.

If you bought a previous edition of the book (for PyQt5, PySide2 or PySide6) you get this update for free! Just log into your account on LearnPyQt and you'll find the book already waiting for you under "My Books & Downloads".

If you have any questions or difficulty getting hold of this update, just get in touch.

Enjoy!

See the complete PyQt5 tutorial, from first steps to complete applications with Python & Qt5.

Categories: FLOSS Project Planets

Quansight Labs Blog: PyTorch TensorIterator Internals - 2021 Update

Planet Python - Fri, 2021-04-09 10:00

For contributors to the PyTorch codebase, one of the most commonly encountered C++ classes is TensorIterator. TensorIterator offers a standardized way to iterate over elements of a tensor, automatically parallelizing operations, while abstracting device and data type details.

In April 2020, Sameer Deshmukh wrote a blog article discussing PyTorch TensorIterator Internals. Recently, however, the interface has changed significantly. This post describes how to use the current interface as of April 2021. Much of the information from the previous article is directly copied here, but with updated API calls and some extra details.

Read more… (8 min remaining to read)

Categories: FLOSS Project Planets

Michael Prokop: A Ceph war story

Planet Debian - Fri, 2021-04-09 08:39

It all started with the big bang! We nearly lost 33 of 36 disks on a Proxmox/Ceph Cluster; this is the story of how we recovered them.

At the end of 2020, we eventually had a long outstanding maintenance window for taking care of system upgrades at a customer. During this maintenance window, which involved reboots of server systems, the involved Ceph cluster unexpectedly went into a critical state. What was planned to be a few hours of checklist work in the early evening turned out to be an emergency case; let’s call it a nightmare (not only because it included a big part of the night). Since we have learned a few things from our post mortem and RCA, it’s worth sharing those with others. But first things first, let’s step back and clarify what we had to deal with.

The system and its upgrade

One part of the upgrade included 3 Debian servers (we’re calling them server1, server2 and server3 here), running on Proxmox v5 + Debian/stretch with 12 Ceph OSDs each (65.45TB in total), a so-called Proxmox Hyper-Converged Ceph Cluster.

First, we went for upgrading the Proxmox v5/stretch system to Proxmox v6/buster, before updating Ceph Luminous v12.2.13 to the latest v14.2 release, supported by Proxmox v6/buster. The Proxmox upgrade included updating corosync from v2 to v3. As part of this upgrade, we had to apply some configuration changes, like adjust ring0 + ring1 address settings and add a mon_host configuration to the Ceph configuration.

During the first two servers’ reboots, we noticed configuration glitches. After fixing those, we went for a reboot of the third server as well. Then we noticed that several Ceph OSDs were unexpectedly down. The NTP service wasn’t working as expected after the upgrade. The underlying issue is a race condition of ntp with systemd-timesyncd (see #889290). As a result, we had clock skew problems with Ceph, indicating that the Ceph monitors’ clocks aren’t running in sync (which is essential for proper Ceph operation). We initially assumed that our Ceph OSD failure derived from this clock skew problem, so we took care of it. After yet another round of reboots, to ensure the systems are running all with identical and sane configurations and services, we noticed lots of failing OSDs. This time all but three OSDs (19, 21 and 22) were down:

% sudo ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 65.44138 root default -2 21.81310 host server1 0 hdd 1.08989 osd.0 down 1.00000 1.00000 1 hdd 1.08989 osd.1 down 1.00000 1.00000 2 hdd 1.63539 osd.2 down 1.00000 1.00000 3 hdd 1.63539 osd.3 down 1.00000 1.00000 4 hdd 1.63539 osd.4 down 1.00000 1.00000 5 hdd 1.63539 osd.5 down 1.00000 1.00000 18 hdd 2.18279 osd.18 down 1.00000 1.00000 20 hdd 2.18179 osd.20 down 1.00000 1.00000 28 hdd 2.18179 osd.28 down 1.00000 1.00000 29 hdd 2.18179 osd.29 down 1.00000 1.00000 30 hdd 2.18179 osd.30 down 1.00000 1.00000 31 hdd 2.18179 osd.31 down 1.00000 1.00000 -4 21.81409 host server2 6 hdd 1.08989 osd.6 down 1.00000 1.00000 7 hdd 1.08989 osd.7 down 1.00000 1.00000 8 hdd 1.63539 osd.8 down 1.00000 1.00000 9 hdd 1.63539 osd.9 down 1.00000 1.00000 10 hdd 1.63539 osd.10 down 1.00000 1.00000 11 hdd 1.63539 osd.11 down 1.00000 1.00000 19 hdd 2.18179 osd.19 up 1.00000 1.00000 21 hdd 2.18279 osd.21 up 1.00000 1.00000 22 hdd 2.18279 osd.22 up 1.00000 1.00000 32 hdd 2.18179 osd.32 down 1.00000 1.00000 33 hdd 2.18179 osd.33 down 1.00000 1.00000 34 hdd 2.18179 osd.34 down 1.00000 1.00000 -3 21.81419 host server3 12 hdd 1.08989 osd.12 down 1.00000 1.00000 13 hdd 1.08989 osd.13 down 1.00000 1.00000 14 hdd 1.63539 osd.14 down 1.00000 1.00000 15 hdd 1.63539 osd.15 down 1.00000 1.00000 16 hdd 1.63539 osd.16 down 1.00000 1.00000 17 hdd 1.63539 osd.17 down 1.00000 1.00000 23 hdd 2.18190 osd.23 down 1.00000 1.00000 24 hdd 2.18279 osd.24 down 1.00000 1.00000 25 hdd 2.18279 osd.25 down 1.00000 1.00000 35 hdd 2.18179 osd.35 down 1.00000 1.00000 36 hdd 2.18179 osd.36 down 1.00000 1.00000 37 hdd 2.18179 osd.37 down 1.00000 1.00000

Our blood pressure increased slightly! Did we just lose all of our cluster? What happened, and how can we get all the other OSDs back?

We stumbled upon this beauty in our logs:

kernel: [ 73.697957] XFS (sdl1): SB stripe unit sanity check failed kernel: [ 73.698002] XFS (sdl1): Metadata corruption detected at xfs_sb_read_verify+0x10e/0x180 [xfs], xfs_sb block 0xffffffffffffffff kernel: [ 73.698799] XFS (sdl1): Unmount and run xfs_repair kernel: [ 73.699199] XFS (sdl1): First 128 bytes of corrupted metadata buffer: kernel: [ 73.699677] 00000000: 58 46 53 42 00 00 10 00 00 00 00 00 00 00 62 00 XFSB..........b. kernel: [ 73.700205] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ kernel: [ 73.700836] 00000020: 62 44 2b c0 e6 22 40 d7 84 3d e1 cc 65 88 e9 d8 bD+.."@..=..e... kernel: [ 73.701347] 00000030: 00 00 00 00 00 00 40 08 00 00 00 00 00 00 01 00 ......@......... kernel: [ 73.701770] 00000040: 00 00 00 00 00 00 01 01 00 00 00 00 00 00 01 02 ................ ceph-disk[4240]: mount: /var/lib/ceph/tmp/mnt.jw367Y: mount(2) system call failed: Structure needs cleaning. ceph-disk[4240]: ceph-disk: Mounting filesystem failed: Command '['/bin/mount', '-t', u'xfs', '-o', 'noatime,inode64', '--', '/dev/disk/by-parttypeuuid/4fbd7e29-9d25-41b8-afd0-062c0ceff05d.cdda39ed-5 ceph/tmp/mnt.jw367Y']' returned non-zero exit status 32 kernel: [ 73.702162] 00000050: 00 00 00 01 00 00 18 80 00 00 00 04 00 00 00 00 ................ kernel: [ 73.702550] 00000060: 00 00 06 48 bd a5 10 00 08 00 00 02 00 00 00 00 ...H............ kernel: [ 73.702975] 00000070: 00 00 00 00 00 00 00 00 0c 0c 0b 01 0d 00 00 19 ................ kernel: [ 73.703373] XFS (sdl1): SB validate failed with error -117.

The same issue was present for the other failing OSDs. We hoped, that the data itself was still there, and only the mounting of the XFS partitions failed. The Ceph cluster was initially installed in 2017 with Ceph jewel/10.2 with the OSDs on filestore (nowadays being a legacy approach to storing objects in Ceph). However, we migrated the disks to bluestore since then (with ceph-disk and not yet via ceph-volume what’s being used nowadays). Using ceph-disk introduces these 100MB XFS partitions containing basic metadata for the OSD.

Given that we had three working OSDs left, we decided to investigate how to rebuild the failing ones. Some folks on #ceph (thanks T1, ormandj + peetaur!) were kind enough to share how working XFS partitions looked like for them. After creating a backup (via dd), we tried to re-create such an XFS partition on server1. We noticed that even mounting a freshly created XFS partition failed:

synpromika@server1 ~ % sudo mkfs.xfs -f -i size=2048 -m uuid="4568c300-ad83-4288-963e-badcd99bf54f" /dev/sdc1 meta-data=/dev/sdc1 isize=2048 agcount=4, agsize=6272 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=0 data = bsize=4096 blocks=25088, imaxpct=25 = sunit=128 swidth=64 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=1608, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 synpromika@server1 ~ % sudo mount /dev/sdc1 /mnt/ceph-recovery SB stripe unit sanity check failed Metadata corruption detected at 0x433840, xfs_sb block 0x0/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0x0/0x1000 cache_node_purge: refcount was 1, not zero (node=0x1d3c400) SB stripe unit sanity check failed Metadata corruption detected at 0x433840, xfs_sb block 0x18800/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0x18800/0x1000 SB stripe unit sanity check failed Metadata corruption detected at 0x433840, xfs_sb block 0x0/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0x0/0x1000 SB stripe unit sanity check failed Metadata corruption detected at 0x433840, xfs_sb block 0x24c00/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0x24c00/0x1000 SB stripe unit sanity check failed Metadata corruption detected at 0x433840, xfs_sb block 0xc400/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0xc400/0x1000 releasing dirty buffer (bulk) to free list!releasing dirty buffer (bulk) to free list!releasing dirty buffer (bulk) to free list!releasing dirty buffer (bulk) to free list!found dirty buffer (bulk) on free list!bad magic number bad magic number Metadata corruption detected at 0x433840, xfs_sb block 0x0/0x1000 libxfs_writebufr: write verifer failed on xfs_sb bno 0x0/0x1000 releasing dirty buffer (bulk) to free list!mount: /mnt/ceph-recovery: wrong fs type, bad option, bad superblock on /dev/sdc1, missing codepage or helper program, or other error.

Ouch. This very much looked related to the actual issue we’re seeing. So we tried to execute mkfs.xfs with a bunch of different sunit/swidth settings. Using ‘-d sunit=512 -d swidth=512‘ at least worked then, so we decided to force its usage in the creation of our OSD XFS partition. This brought us a working XFS partition. Please note, sunit must not be larger than swidth (more on that later!).

Then we reconstructed how to restore all the metadata for the OSD (activate.monmap, active, block_uuid, bluefs, ceph_fsid, fsid, keyring, kv_backend, magic, mkfs_done, ready, require_osd_release, systemd, type, whoami). To identify the UUID, we can read the data from ‘ceph --format json osd dump‘, like this for all our OSDs (Zsh syntax ftw!):

synpromika@server1 ~ % for f in {0..37} ; printf "osd-$f: %s\n" "$(sudo ceph --format json osd dump | jq -r ".osds[] | select(.osd==$f) | .uuid")" osd-0: 4568c300-ad83-4288-963e-badcd99bf54f osd-1: e573a17a-ccde-4719-bdf8-eef66903ca4f osd-2: 0e1b2626-f248-4e7d-9950-f1a46644754e osd-3: 1ac6a0a2-20ee-4ed8-9f76-d24e900c800c [...]

Identifying the corresponding raw device for each OSD UUID is possible via:

synpromika@server1 ~ % UUID="4568c300-ad83-4288-963e-badcd99bf54f" synpromika@server1 ~ % readlink -f /dev/disk/by-partuuid/"${UUID}" /dev/sdc1

The OSD’s key ID can be retrieved via:

synpromika@server1 ~ % OSD_ID=0 synpromika@server1 ~ % sudo ceph auth get osd."${OSD_ID}" -f json 2>/dev/null | jq -r '.[] | .key' AQCKFpZdm0We[...]

Now we also need to identify the underlying block device:

synpromika@server1 ~ % OSD_ID=0 synpromika@server1 ~ % sudo ceph osd metadata osd."${OSD_ID}" -f json | jq -r '.bluestore_bdev_partition_path' /dev/sdc2

With all of this, we reconstructed the keyring, fsid, whoami, block + block_uuid files. All the other files inside the XFS metadata partition are identical on each OSD. So after placing and adjusting the corresponding metadata on the XFS partition for Ceph usage, we got a working OSD – hurray! Since we had to fix yet another 32 OSDs, we decided to automate this XFS partitioning and metadata recovery procedure.

We had a network share available on /srv/backup for storing backups of existing partition data. On each server, we tested the procedure with one single OSD before iterating over the list of remaining failing OSDs. We started with a shell script on server1, then adjusted the script for server2 and server3. This is the script, as we executed it on the 3rd server.

Thanks to this, we managed to get the Ceph cluster up and running again. We didn’t want to continue with the Ceph upgrade itself during the night though, as we wanted to know exactly what was going on and why the system behaved like that. Time for RCA!

Root Cause Analysis

So all but three OSDs on server2 failed, and the problem seems to be related to XFS. Therefore, our starting point for the RCA was, to identify what was different on server2, as compared to server1 + server3. My initial assumption was that this was related to some firmware issues with the involved controller (and as it turned out later, I was right!). The disks were attached as JBOD devices to a ServeRAID M5210 controller (with a stripe size of 512). Firmware state:

synpromika@server1 ~ % sudo storcli64 /c0 show all | grep '^Firmware' Firmware Package Build = 24.16.0-0092 Firmware Version = 4.660.00-8156 synpromika@server2 ~ % sudo storcli64 /c0 show all | grep '^Firmware' Firmware Package Build = 24.21.0-0112 Firmware Version = 4.680.00-8489 synpromika@server3 ~ % sudo storcli64 /c0 show all | grep '^Firmware' Firmware Package Build = 24.16.0-0092 Firmware Version = 4.660.00-8156

This looked very promising, as server2 indeed runs with a different firmware version on the controller. But how so? Well, the motherboard of server2 got replaced by a Lenovo/IBM technician in January 2020, as we had a failing memory slot during a memory upgrade. As part of this procedure, the Lenovo/IBM technician installed the latest firmware versions. According to our documentation, some OSDs were rebuilt (due to the filestore->bluestore migration) in March and April 2020. It turned out that precisely those OSDs were the ones that survived the upgrade. So the surviving drives were created with a different firmware version running on the involved controller. All the other OSDs were created with an older controller firmware. But what difference does this make?

Now let’s check firmware changelogs. For the 24.21.0-0097 release we found this:

- Cannot create or mount xfs filesystem using xfsprogs 4.19.x kernel 4.20(SCGCQ02027889) - xfs_info command run on an XFS file system created on a VD of strip size 1M shows sunit and swidth as 0(SCGCQ02056038)

Our XFS problem certainly was related to the controller’s firmware. We also recalled that our monitoring system reported different sunit settings for the OSDs that were rebuilt in March and April. For example, OSD 21 was recreated and got different sunit settings:

WARN server2.example.org Mount options of /var/lib/ceph/osd/ceph-21 WARN - Missing: sunit=1024, Exceeding: sunit=512

We compared the new OSD 21 with an existing one (OSD 25 on server3):

synpromika@server2 ~ % systemctl show var-lib-ceph-osd-ceph\\x2d21.mount | grep sunit Options=rw,noatime,attr2,inode64,sunit=512,swidth=512,noquota synpromika@server3 ~ % systemctl show var-lib-ceph-osd-ceph\\x2d25.mount | grep sunit Options=rw,noatime,attr2,inode64,sunit=1024,swidth=512,noquota

Thanks to our documentation, we could compare execution logs of their creation:

% diff -u ceph-disk-osd-25.log ceph-disk-osd-21.log -synpromika@server2 ~ % sudo ceph-disk -v prepare --bluestore /dev/sdj --osd-id 25 +synpromika@server3 ~ % sudo ceph-disk -v prepare --bluestore /dev/sdi --osd-id 21 [...] -command_check_call: Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdj1 -meta-data=/dev/sdj1 isize=2048 agcount=4, agsize=6272 blks [...] +command_check_call: Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdi1 +meta-data=/dev/sdi1 isize=2048 agcount=4, agsize=6336 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 -data = bsize=4096 blocks=25088, imaxpct=25 - = sunit=128 swidth=64 blks +data = bsize=4096 blocks=25344, imaxpct=25 + = sunit=64 swidth=64 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=1608, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 [...]

So back then, we even tried to track this down but couldn’t make sense of it yet. But now this sounds very much like it is related to the problem we saw with this Ceph/XFS failure. We follow Occam’s razor, assuming the simplest explanation is usually the right one, so let’s check the disk properties and see what differs:

synpromika@server1 ~ % sudo blockdev --getsz --getsize64 --getss --getpbsz --getiomin --getioopt /dev/sdk 4685545472 2398999281664 512 4096 524288 262144 synpromika@server2 ~ % sudo blockdev --getsz --getsize64 --getss --getpbsz --getiomin --getioopt /dev/sdk 4685545472 2398999281664 512 4096 262144 262144

See the difference between server1 and server2 for identical disks? The getiomin option now reports something different for them:

synpromika@server1 ~ % sudo blockdev --getiomin /dev/sdk 524288 synpromika@server1 ~ % cat /sys/block/sdk/queue/minimum_io_size 524288 synpromika@server2 ~ % sudo blockdev --getiomin /dev/sdk 262144 synpromika@server2 ~ % cat /sys/block/sdk/queue/minimum_io_size 262144

It doesn’t make sense that the minimum I/O size (iomin, AKA BLKIOMIN) is bigger than the optimal I/O size (ioopt, AKA BLKIOOPT). This leads us to Bug 202127 – cannot mount or create xfs on a 597T device, which matches our findings here. But why did this XFS partition work in the past and fails now with the newer kernel version?

The XFS behaviour change

Now given that we have backups of all the XFS partition, we wanted to track down, a) when this XFS behaviour was introduced, and b) whether, and if so how it would be possible to reuse the XFS partition without having to rebuild it from scratch (e.g. if you would have no working Ceph OSD or backups left).

Let’s look at such a failing XFS partition with the Grml live system:

root@grml ~ # grml-version grml64-full 2020.06 Release Codename Ausgehfuahangl [2020-06-24] root@grml ~ # uname -a Linux grml 5.6.0-2-amd64 #1 SMP Debian 5.6.14-2 (2020-06-09) x86_64 GNU/Linux root@grml ~ # grml-hostname grml-2020-06 Setting hostname to grml-2020-06: done root@grml ~ # exec zsh root@grml-2020-06 ~ # dpkg -l xfsprogs util-linux Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-========================================= ii util-linux 2.35.2-4 amd64 miscellaneous system utilities ii xfsprogs 5.6.0-1+b2 amd64 Utilities for managing the XFS filesystem

There it’s failing, no matter which mount option we try:

root@grml-2020-06 ~ # mount ./sdd1.dd /mnt mount: /mnt: mount(2) system call failed: Structure needs cleaning. root@grml-2020-06 ~ # dmesg | tail -30 [...] [ 64.788640] XFS (loop1): SB stripe unit sanity check failed [ 64.788671] XFS (loop1): Metadata corruption detected at xfs_sb_read_verify+0x102/0x170 [xfs], xfs_sb block 0xffffffffffffffff [ 64.788671] XFS (loop1): Unmount and run xfs_repair [ 64.788672] XFS (loop1): First 128 bytes of corrupted metadata buffer: [ 64.788673] 00000000: 58 46 53 42 00 00 10 00 00 00 00 00 00 00 62 00 XFSB..........b. [ 64.788674] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 64.788675] 00000020: 32 b6 dc 35 53 b7 44 96 9d 63 30 ab b3 2b 68 36 2..5S.D..c0..+h6 [ 64.788675] 00000030: 00 00 00 00 00 00 40 08 00 00 00 00 00 00 01 00 ......@......... [ 64.788675] 00000040: 00 00 00 00 00 00 01 01 00 00 00 00 00 00 01 02 ................ [ 64.788676] 00000050: 00 00 00 01 00 00 18 80 00 00 00 04 00 00 00 00 ................ [ 64.788677] 00000060: 00 00 06 48 bd a5 10 00 08 00 00 02 00 00 00 00 ...H............ [ 64.788677] 00000070: 00 00 00 00 00 00 00 00 0c 0c 0b 01 0d 00 00 19 ................ [ 64.788679] XFS (loop1): SB validate failed with error -117. root@grml-2020-06 ~ # mount -t xfs -o rw,relatime,attr2,inode64,sunit=1024,swidth=512,noquota ./sdd1.dd /mnt/ mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop1, missing codepage or helper program, or other error. 32 root@grml-2020-06 ~ # dmesg | tail -1 [ 66.342976] XFS (loop1): stripe width (512) must be a multiple of the stripe unit (1024) root@grml-2020-06 ~ # mount -t xfs -o rw,relatime,attr2,inode64,sunit=512,swidth=512,noquota ./sdd1.dd /mnt/ mount: /mnt: mount(2) system call failed: Structure needs cleaning. 32 root@grml-2020-06 ~ # dmesg | tail -14 [ 66.342976] XFS (loop1): stripe width (512) must be a multiple of the stripe unit (1024) [ 80.751277] XFS (loop1): SB stripe unit sanity check failed [ 80.751323] XFS (loop1): Metadata corruption detected at xfs_sb_read_verify+0x102/0x170 [xfs], xfs_sb block 0xffffffffffffffff [ 80.751324] XFS (loop1): Unmount and run xfs_repair [ 80.751325] XFS (loop1): First 128 bytes of corrupted metadata buffer: [ 80.751327] 00000000: 58 46 53 42 00 00 10 00 00 00 00 00 00 00 62 00 XFSB..........b. [ 80.751328] 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 80.751330] 00000020: 32 b6 dc 35 53 b7 44 96 9d 63 30 ab b3 2b 68 36 2..5S.D..c0..+h6 [ 80.751331] 00000030: 00 00 00 00 00 00 40 08 00 00 00 00 00 00 01 00 ......@......... [ 80.751331] 00000040: 00 00 00 00 00 00 01 01 00 00 00 00 00 00 01 02 ................ [ 80.751332] 00000050: 00 00 00 01 00 00 18 80 00 00 00 04 00 00 00 00 ................ [ 80.751333] 00000060: 00 00 06 48 bd a5 10 00 08 00 00 02 00 00 00 00 ...H............ [ 80.751334] 00000070: 00 00 00 00 00 00 00 00 0c 0c 0b 01 0d 00 00 19 ................ [ 80.751338] XFS (loop1): SB validate failed with error -117.

Also xfs_repair doesn’t help either:

root@grml-2020-06 ~ # xfs_info ./sdd1.dd meta-data=./sdd1.dd isize=2048 agcount=4, agsize=6272 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=0, rmapbt=0 = reflink=0 data = bsize=4096 blocks=25088, imaxpct=25 = sunit=128 swidth=64 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=1608, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 root@grml-2020-06 ~ # xfs_repair ./sdd1.dd Phase 1 - find and verify superblock... bad primary superblock - bad stripe width in superblock !!! attempting to find secondary superblock... ..............................................................................................Sorry, could not find valid secondary superblock Exiting now.

With the “SB stripe unit sanity check failed” message, we could easily track this down to the following commit fa4ca9c:

% git show fa4ca9c5574605d1e48b7e617705230a0640b6da | cat commit fa4ca9c5574605d1e48b7e617705230a0640b6da Author: Dave Chinner <dchinner@redhat.com> Date: Tue Jun 5 10:06:16 2018 -0700 xfs: catch bad stripe alignment configurations When stripe alignments are invalid, data alignment algorithms in the allocator may not work correctly. Ensure we catch superblocks with invalid stripe alignment setups at mount time. These data alignment mismatches are now detected at mount time like this: XFS (loop0): SB stripe unit sanity check failed XFS (loop0): Metadata corruption detected at xfs_sb_read_verify+0xab/0x110, xfs_sb block 0xffffffffffffffff XFS (loop0): Unmount and run xfs_repair XFS (loop0): First 128 bytes of corrupted metadata buffer: 0000000091c2de02: 58 46 53 42 00 00 10 00 00 00 00 00 00 00 10 00 XFSB............ 0000000023bff869: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00000000cdd8c893: 17 32 37 15 ff ca 46 3d 9a 17 d3 33 04 b5 f1 a2 .27...F=...3.... 000000009fd2844f: 00 00 00 00 00 00 00 04 00 00 00 00 00 00 06 d0 ................ 0000000088e9b0bb: 00 00 00 00 00 00 06 d1 00 00 00 00 00 00 06 d2 ................ 00000000ff233a20: 00 00 00 01 00 00 10 00 00 00 00 01 00 00 00 00 ................ 000000009db0ac8b: 00 00 03 60 e1 34 02 00 08 00 00 02 00 00 00 00 ...`.4.......... 00000000f7022460: 00 00 00 00 00 00 00 00 0c 09 0b 01 0c 00 00 19 ................ XFS (loop0): SB validate failed with error -117. And the mount fails. Signed-off-by: Dave Chinner <dchinner@redhat.com> Reviewed-by: Carlos Maiolino <cmaiolino@redhat.com> Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> diff --git fs/xfs/libxfs/xfs_sb.c fs/xfs/libxfs/xfs_sb.c index b5dca3c8c84d..c06b6fc92966 100644 --- fs/xfs/libxfs/xfs_sb.c +++ fs/xfs/libxfs/xfs_sb.c @@ -278,6 +278,22 @@ xfs_mount_validate_sb( return -EFSCORRUPTED; } + if (sbp->sb_unit) { + if (!xfs_sb_version_hasdalign(sbp) || + sbp->sb_unit > sbp->sb_width || + (sbp->sb_width % sbp->sb_unit) != 0) { + xfs_notice(mp, "SB stripe unit sanity check failed"); + return -EFSCORRUPTED; + } + } else if (xfs_sb_version_hasdalign(sbp)) { + xfs_notice(mp, "SB stripe alignment sanity check failed"); + return -EFSCORRUPTED; + } else if (sbp->sb_width) { + xfs_notice(mp, "SB stripe width sanity check failed"); + return -EFSCORRUPTED; + } + + if (xfs_sb_version_hascrc(&mp->m_sb) && sbp->sb_blocksize < XFS_MIN_CRC_BLOCKSIZE) { xfs_notice(mp, "v5 SB sanity check failed");

This change is included in kernel versions 4.18-rc1 and newer:

% git describe --contains fa4ca9c5574605d1e48 v4.18-rc1~37^2~14

Now let’s try with an older kernel version (4.9.0), using old Grml 2017.05 release:

root@grml ~ # grml-version grml64-small 2017.05 Release Codename Freedatensuppe [2017-05-31] root@grml ~ # uname -a Linux grml 4.9.0-1-grml-amd64 #1 SMP Debian 4.9.29-1+grml.1 (2017-05-24) x86_64 GNU/Linux root@grml ~ # lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 9.0 (stretch) Release: 9.0 Codename: stretch root@grml ~ # grml-hostname grml-2017-05 Setting hostname to grml-2017-05: done root@grml ~ # exec zsh root@grml-2017-05 ~ # root@grml-2017-05 ~ # xfs_info ./sdd1.dd xfs_info: ./sdd1.dd is not a mounted XFS filesystem 1 root@grml-2017-05 ~ # xfs_repair ./sdd1.dd Phase 1 - find and verify superblock... bad primary superblock - bad stripe width in superblock !!! attempting to find secondary superblock... ..............................................................................................Sorry, could not find valid secondary superblock Exiting now. 1 root@grml-2017-05 ~ # mount ./sdd1.dd /mnt root@grml-2017-05 ~ # mount -t xfs /root/sdd1.dd on /mnt type xfs (rw,relatime,attr2,inode64,sunit=1024,swidth=512,noquota) root@grml-2017-05 ~ # ls /mnt activate.monmap active block block_uuid bluefs ceph_fsid fsid keyring kv_backend magic mkfs_done ready require_osd_release systemd type whoami root@grml-2017-05 ~ # xfs_info /mnt meta-data=/dev/loop1 isize=2048 agcount=4, agsize=6272 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1 spinodes=0 rmapbt=0 = reflink=0 data = bsize=4096 blocks=25088, imaxpct=25 = sunit=128 swidth=64 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal bsize=4096 blocks=1608, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0

Mounting there indeed works! Now, if we mount the filesystem with new and proper sunit/swidth settings using the older kernel, it should rewrite them on disk:

root@grml-2017-05 ~ # mount -t xfs -o sunit=512,swidth=512 ./sdd1.dd /mnt/ root@grml-2017-05 ~ # umount /mnt/

And indeed, mounting this rewritten filesystem then also works with newer kernels:

root@grml-2020-06 ~ # mount ./sdd1.rewritten /mnt/ root@grml-2020-06 ~ # xfs_info /root/sdd1.rewritten meta-data=/dev/loop1 isize=2048 agcount=4, agsize=6272 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=0, rmapbt=0 = reflink=0 data = bsize=4096 blocks=25088, imaxpct=25 = sunit=64 swidth=64 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=1608, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 root@grml-2020-06 ~ # mount -t xfs /root/sdd1.rewritten on /mnt type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,sunit=512,swidth=512,noquota)

FTR: The ‘sunit=512,swidth=512‘ from the xfs mount option is identical to xfs_info’s output ‘sunit=64,swidth=64‘ (because mount.xfs’s sunit value is given in 512-byte block units, see man 5 xfs, and the xfs_info output reported here is in blocks with a block size (bsize) of 4096, so ‘sunit = 512*512 := 64*4096‘).

mkfs uses minimum and optimal sizes for stripe unit and stripe width; you can check this e.g. via (note that server2 with fixed firmware version reports proper values, whereas server3 with broken controller firmware reports non-sense):

synpromika@server2 ~ % for i in /sys/block/sd*/queue/ ; do printf "%s: %s %s\n" "$i" "$(cat "$i"/minimum_io_size)" "$(cat "$i"/optimal_io_size)" ; done [...] /sys/block/sdc/queue/: 262144 262144 /sys/block/sdd/queue/: 262144 262144 /sys/block/sde/queue/: 262144 262144 /sys/block/sdf/queue/: 262144 262144 /sys/block/sdg/queue/: 262144 262144 /sys/block/sdh/queue/: 262144 262144 /sys/block/sdi/queue/: 262144 262144 /sys/block/sdj/queue/: 262144 262144 /sys/block/sdk/queue/: 262144 262144 /sys/block/sdl/queue/: 262144 262144 /sys/block/sdm/queue/: 262144 262144 /sys/block/sdn/queue/: 262144 262144 [...] synpromika@server3 ~ % for i in /sys/block/sd*/queue/ ; do printf "%s: %s %s\n" "$i" "$(cat "$i"/minimum_io_size)" "$(cat "$i"/optimal_io_size)" ; done [...] /sys/block/sdc/queue/: 524288 262144 /sys/block/sdd/queue/: 524288 262144 /sys/block/sde/queue/: 524288 262144 /sys/block/sdf/queue/: 524288 262144 /sys/block/sdg/queue/: 524288 262144 /sys/block/sdh/queue/: 524288 262144 /sys/block/sdi/queue/: 524288 262144 /sys/block/sdj/queue/: 524288 262144 /sys/block/sdk/queue/: 524288 262144 /sys/block/sdl/queue/: 524288 262144 /sys/block/sdm/queue/: 524288 262144 /sys/block/sdn/queue/: 524288 262144 [...]

This is the underlying reason why the initially created XFS partitions were created with incorrect sunit/swidth settings. The broken firmware of server1 and server3 was the cause of the incorrect settings – they were ignored by old(er) xfs/kernel versions, but treated as an error by new ones.

Make sure to also read the XFS FAQ regarding “How to calculate the correct sunit,swidth values for optimal performance”. We also stumbled upon two interesting reads in RedHat’s knowledge base: 5075561 + 2150101 (requires an active subscription, though) and #1835947.

Am I affected? How to work around it?

To check whether your XFS mount points are affected by this issue, the following command line should be useful:

awk '$3 == "xfs"{print $2}' /proc/self/mounts | while read mount ; do echo -n "$mount " ; xfs_info $mount | awk '$0 ~ "swidth"{gsub(/.*=/,"",$2); gsub(/.*=/,"",$3); print $2,$3}' | awk '{ if ($1 > $2) print "impacted"; else print "OK"}' ; done

If you run into the above situation, the only known solution to get your original XFS partition working again, is to boot into an older kernel version again (4.17 or older), mount the XFS partition with correct sunit/swidth settings and then boot back into your new system (kernel version wise).

Lessons learned
  • document everything and ensure to have all relevant information available (including actual times of changes, used kernel/package/firmware/… versions. The thorough documentation was our most significant asset in this case, because we had all the data and information we needed during the emergency handling as well as for the post mortem/RCA)
  • if something changes unexpectedly, dig deeper
  • know who to ask, a network of experts pays off
  • including timestamps in your shell makes reconstruction easier (the more people and documentation involved, the harder it gets to wade through it)
  • keep an eye on changelogs/release notes
  • apply regular updates and don’t forget invisible layers (e.g. BIOS, controller/disk firmware, IPMI/OOB (ILO/RAC/IMM/…) firmware)
  • apply regular reboots, to avoid a possible delta becoming bigger (which makes debugging harder)

Thanks: Darshaka Pathirana, Chris Hofstaedtler and Michael Hanscho.

Looking for help with your IT infrastructure? Let us know!

Categories: FLOSS Project Planets

Real Python: The Real Python Podcast – Episode #55: Getting Started With Refactoring Your Python Code

Planet Python - Fri, 2021-04-09 08:00

Do you think it's time to refactor your Python code? What should you think about before starting this task? This week on the show, we have Brendan Maginnis and Nick Thapen from Sourcery. Sourcery is an automated refactoring tool that integrates into your IDE and suggests improvements to your code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Categories: FLOSS Project Planets

Python for Beginners: Convert a list containing float numbers to string in Python

Planet Python - Fri, 2021-04-09 07:58

There may be situations where we want to convert a list containing float numbers to string while working in python. In this article, we will look at different ways to convert the list containing float numbers to a string which contains the elements of the list as space separated sub strings.

Important functions for the task to convert a list containing float numbers to string

To convert a list of floating point numbers to string, we will need several string methods which we will discuss first and then perform the operations.

str() function

The str() function converts an integer literal,a float literal or any other given  input to a string literal and returns the string literal of the input after conversion. This can be done as follows.

float_num=10.0 str_num=str(float_num)

join() method

join() method is invoked on a separator and an iterable object of strings is passed to the method as input. It joins each string in the iterable object with the separator and returns a new string. 

The syntax for join() method is as separator.join(iterable) where separator can be any character and iterable can be a list or tuple etc. This can be understood as follows.

str_list=["I","am","Python","String"] print("List of strings is:") print(str_list) separator=" " str_output=separator.join(str_list) print("String output is:") print(str_output)

Output:

List of strings is: ['I', 'am', 'Python', 'String'] String output is: I am Python String

map() function

The map() function takes as input a function(functions are first class objects and can be passed as a parameter in python) and an iterable as argument and executes the function on each element of the iterable object and returns  the output map object which can be converted into any iterable.

The syntax for map() function is map(function,iterable). This can be understood as follows.

float_list=[10.0,11.2,11.7,12.1] print("List of floating point numbers is:") print(float_list) str_list=list(map(str,float_list)) print("List of String numbers is:") print(str_list)

Output:

List of floating point numbers is: [10.0, 11.2, 11.7, 12.1] List of String numbers is: ['10.0', '11.2', '11.7', '12.1']

Now we will see how to convert a list containing floating point numbers to string using the above functions.

Convert a list containing float numbers to string using for loop

We can convert the list of float numbers to string by declaring an empty string and then performing string concatenation to add the elements of the list to the string as follows.

float_list=[10.0,11.2,11.7,12.1] print("List of floating point numbers is:") print(float_list) float_string="" for num in float_list: float_string=float_string+str(num)+" " print("Output String is:") print(float_string.rstrip())

Output:

List of floating point numbers is: [10.0, 11.2, 11.7, 12.1] Output String is: 10.0 11.2 11.7 12.1

 In the above program, the output string contains an extra space at the end which has to be removed using rstrip() method.To avoid this additional operation, We can use the join() method instead of performing concatenation operation by adding the strings to create the output string as follows.

float_list=[10.0,11.2,11.7,12.1] print("List of floating point numbers is:") print(float_list) float_string="" for num in float_list: float_string=" ".join([float_string,str(num)]) print("Output String is:") print(float_string.lstrip())

Output

List of floating point numbers is: [10.0, 11.2, 11.7, 12.1] Output String is: 10.0 11.2 11.7 12.1

In the above method, an extra space is added at left of the output string which has to be removed using lstrip() method. To avoid this,Instead of applying str() function on every element of the list, we can use map() function  to convert the list of float numbers to a list of strings and then perform the string concatenation using join() method to get the output string as follows.

float_list=[10.0,11.2,11.7,12.1] print("List of floating point numbers is:") print(float_list) str_list=list(map(str,float_list)) print("List of String numbers is:") print(str_list) float_string="" for num in float_list: float_string=" ".join(str_list) print("Output String is:") print(float_string)

Output:

List of floating point numbers is: [10.0, 11.2, 11.7, 12.1] List of String numbers is: ['10.0', '11.2', '11.7', '12.1'] Output String is: 10.0 11.2 11.7 12.1 Convert a list containing float numbers to string using list comprehension

Instead of for loop, we can use list comprehension to perform the conversion of a list of floating point numbers to string as follows. We can use the join() method with list comprehension as follows.

float_list=[10.0,11.2,11.7,12.1] print("List of floating point numbers is:") print(float_list) str_list=[str(i) for i in float_list] print("List of floating string numbers is:") print(str_list) float_string=" ".join(str_list) print("Output String is:") print(float_string)

Output:

List of floating point numbers is: [10.0, 11.2, 11.7, 12.1] List of floating string numbers is: ['10.0', '11.2', '11.7', '12.1'] Output String is: 10.0 11.2 11.7 12.1 Conclusion

In this article, we have seen how to convert a list containing floating point numbers into string using different string methods like str(), join() and for loop or list comprehension in python. Stay tuned for more informative articles.

The post Convert a list containing float numbers to string in Python appeared first on PythonForBeginners.com.

Categories: FLOSS Project Planets

Advanced Comic Book Linking

Planet KDE - Fri, 2021-04-09 06:44

In my previous post about Peruse's support for ACBF Textareas, i talked about how the formatting system there allowed for all manner of niftiness with languages, and rotating text, and making it coloured, and styled, and all of that visual fanciness. In this one, i'm going to talk a bit more in depth about a quite powerful feature the Textareas have that i kind of glossed over in that one: Hyperlinks.

Yes, hyperlinks. Links, like the one you just clicked on above to work out what that last post actually said about textareas (thanks for reading! :) ). Except that these links are inside any of the things in an ACBF document which can hold paragraphs of text, being primarily Textareas, References, and Annotations. The reason this is a separate post (apart from the fact the other one had a different focus) is that ACBF recently gained the ability to add a resource target (that is to say, a href property) to the Jump element.

Linking to some URL, and protecting people

For a simple starter, the most usual suspects for hyperlinks work for this: Websites (http(s)://) and email links (mailto:), primarily. If you click on one of those in Peruse, don't worry, we're not going to send you anywhere without making sure you want to actually go there. On the web, you usually would be expected to trust whatever you're clicking on, but inside a book, you'd reasonably expect to be constrained, so before sending you anywhere that isn't inside the book, you get a cute little popup that shows you the address of the link you clicked on, and asks if you really do want to follow it.

Additionally, and very possibly more interesting when creating a book: Internal links work as well. If an object in the book has been given an ID (which is now also more widely possible), you can link to it by prepending it with a # and using that as your link target (which is similar to how you might link to a named anchor in a simple html document). In its simplest form, this allows you to jump from page to page (which also is what the Jump element was originally designed to do), but with this system, much more power is added, by being able to link to not only pages, but also individual frames, and the reference and binary objects.

Linking to a page or a frame will make Peruse navigate to that location in the book, as you would expect (the way Jumps previously would link directly to a page). The idea there is to allow you to create branching narratives and other interactions of that type, like those you would find in a Choose Your Own Adventure book. Of course, ACBF is not a scripting system, and Peruse doesn't have one either; this is "just" markup, so it's not like you can create a full blown game here, in the modern sense, but even then, engaging narratives do not require score keeping for their interest :D

Say you've got a lovely CC-BY banner, and want to show the license? Pop the text in a Reference, and Jump away!

Linking to a reference or a binary object will make Peruse display that object in a popup or a page. An author can use this to create a glossary of terms, for example, or character reference art, a location map, or a world encyclopedia built right into the book itself. As someone who does writing and world building in their spare time, this in itself is something that i'm hoping to expand on greatly (how does having a pile of world-building focused templates for references sound to you? Sounds kind of neat to me, and i'd really like to see that happen :) ).

Something else Peruse does is keep track of all these internal links. Or, rather, the libacbf code does - we built that library as a non-gui Qt style library to be consumable without being bound to Peruse, but more on that at some later date (Tier 1 ACBF KDE Framework anybody? ;) ). What that means is that if you are at a location in a book which is linked to by something else, Peruse knows about it.

Right now, that information isn't used, but we might imagine things like building two-way navigation in the book (say you click a link, and you want to go back to where you came from; well with this we can make sure that happens), or even building entire navigation maps for a book (technically this tracking system for the internal links constitutes a directional graph, where each node is a location in the book, and the edges are the links). How such a thing would be presented visually, i'm frankly unsure, but as a basic idea it feels like it could be fun!

If this sounds like something you'd like to work with, get in touch - either here, or anywhere else you find me - i'm leinir basically everywhere, and we've got our own little corner over on Matrix as well. In short, get in touch, we'd love to hear from you! :)

The word of the day is: Emergent. Because that's the kind of behaviour these bits of functionality promises

Categories: FLOSS Project Planets

Pages