Feeds
Sahil Dhiman: How I Write Blogs - June 2024 Edition
I wrote about my blog writing methodology back April 2021. My writing method has undergone a significant shift now, so thought about writing this update.
New blog topics are added to my note-taking app quite frequently now. Occasionally going through the list, I merge topics, change order to prioritize certain topics or purely drop ideas which seems not worth a write-up. Due to this, I have the liberty to work on blogs according to mood. Writing the last one was tiring, so I chose to work on an easy one, i.e. this blog now.
Topic decided, everything starts on etherpad now. Etherpad has this nice font and sync feature, which helps me write from any device of choice. Actual writing usually happens in the morning, right after I wake up. For most topics, I quickly jot down pointers and keep on expanding them over the course of multiple days at a leisurely pace. Though, sometime it adds too many pieces in the puzzle and takes additional time to put everything in flow. New pointer addition keeps on happening along with writing. Nowadays, pictures too dot my blog, which I rarely use to do earlier. I have come to believe on less usage of external links. These breaks the flow of readers. If someone is sufficiently motivated to learn more about something, finding useful sources isn’t.
As the first draft comes into being, I run it through LanguageTool for spelling corrections (which typically are many) and fixing grammatical issues. Post that, for the first time I read the complete write-up in one go for formation of coherent storyline, moving paragraphs around for form a structure , adding explainers wherever something new or unexplained is introduced, removing elaborate sentences, making amends wherever required and moving paragraphs around for forming structure. Another round of LanguageTool follows. All set now, I try to space out my final read before publishing, which helps find additional mistakes or loopholes.
When everything is set, I do hugo to generate the site and rsync everything to the web server. A final git sync closes the publication part.
After a day or two, I come back to read the blog on the website. This entails another round finding and fixing trivial mistakes. After this, it’s set for good.
Nowadays, in addition to being on my blog, everything is syndicated on Planet FSCI and Planet Debian, which has given it more visibility. As someone who’s into infrastructure and Internet as a lot, I do pay attention to logs on my server, but as a disconnected exercise to if the blog is being read or not. More hits on the blog doesn’t translate to any gratification for me, at least for writers’ point of view. Occasionally, people do mention my blog, which does flatter me. Four years and nearly a hundred posts later, I still wonder how I kept on writing for this long.
Brett Cannon: My impressions of ReScript
I maintain a GitHub Action called check-for-changed-files. For the purpose of this blog post what the action does isn&apost important, but the fact that I authored it originally in TypeScript is. See, one day I tried to update the NPM dependencies. Unfortunately, that update broke everything in a really bad way due to how the libraries I used to access PR details changed and howthe TypeScript types changed. I had also gotten tired of updating the NPM dependencies for security concerns I didn&apost have since this code was only run in CI by others for their own use (i.e. regex denial-of-service isn&apost a big concern). As such I was getting close to burning out on the project as it was a nothing but a chore to keep it up-to-date and I wasn&apost motivated to keep the code up-to-date since TypeScript felt more like a cost than a benefit for such a small code base where I&aposm the sole maintainer (there&aposs only been one other contributor to the project since the initial commit 4.5 years ago). I converted the code base to JavaScript in hopes of simplifying my life and it went better than I expected, but it still wasn&apost enough to keep me interested in the project.
And so I did what I needed to in order to be engaged with the project again: I rewrote it in another programming language that could run easily under Node. 😁 I decided I wanted to do the rewrite piecemeal to make sure I could tell if I was going to like the eventual outcome quickly rather than a complete rewrite from scratch and being unhappy with where I ended up (doing this while on parental leave made me prioritize my spare team immensely, so failing fast was tantamount). During my parental leave I learned Gleam because I loved their statement on expectations for community conduct on their homepage, but while it does compile to JavaScript I realized it works better when JavaScript is used as an escape hatch instead using Gleam to port an existing code base and so it wasn&apost a good fit for this use case.
My next language to attempt the rewrite with was ReScript thanks to my friend Dusty liking it. One of the first things I liked about the language was it had a clear migration path from JavaScript to ReScript in 5 easy steps. And since step 1 was "wrap your JavaScript code in %%raw blocks and change nothing" and step 5 was the optional "clean up" step, there was really only 3 main steps (I did have a hiccup with step 1, though, due to a bug not escaping backticks for template literals appropriately, but it was a mostly mechanical change to undo the template literals and switch to string concatenation).
A key thing that drew me to the language is its OCaml history. ReScript can have very strict typing, but ReScript&aposs OCaml background also means there&aposs type inference, so the typing doesn&apost feel that heavy. ReScript also has a functional programming leaning which I appreciate.
💡When people say "ML" for "machine learning" it still throws me as I instinctively think they are actually referring to "Standard ML".But having said all of that, ReScript does realize folks will be migrating or working with a preexisting JavaScript code base or libraries, and so it tries to be pragmatic for that situation. For instance, while the language has roots in OCaml, the syntax would feel comfortable to JavaScript developers. While supporting a functional style of programming, the language still has things like if/else and for loops. While the language is strongly typed, ReScript as things like its object type where the types of the fields can be inferred based on usage to make it easier to bring over JavaScript objects.
As part of the rewrite I decided to lean in on testing to help make sure things worked as I expected them to. But I ran into an issue where the first 3 testing frameworks I looked into didn&apost work with ReScript 11 (which came out in January 2024 and is the latest major version as I write this). Luckily the 4th one, rescript-zora, worked without issue (it also happens to be by my friend, Dusty, so I was able to ask questions of the author directly 😁; I initially avoided it so I wouldn&apost pester him about stuff, but I made up for it by contributing back). Since ReScript&aposs community isn&apost massive it isn&apost unexpected to have some delays in projects keeping up with stuff. Luckily the ReScript forum is active so you can get your questions answered quickly if you get stuck. But this hiccup and the one involving %%raw and template literals, the process was overall rather smooth.
In the end I would say the experience was a good one. I liked the language and transitioning from JavaScript to ReScript went relatively smoothly. As such, I have ported check-for-changed-files over to ReScript permanently in the 1.2.1 release, and hopefully no one noticed the switch. 🤞
KDE PIM Sprint June 2024
Last weekend I visited Toulouse for the annual KDE PIM sprint. Besides discussing many topics and getting a lot of work done this being in France also had some culinary benefits.
Photo by Carl Schwan. SprintThere’s several reports already on Planet KDE about what we did:
And there’s also the meeting notes in the wiki.
So instead of repeating that I’ll just focus on a few things I worked on. What I’ll repeat though is their thanks to Étincelle Coworking for providing the venue and to Kévin for organizing everything.
Getting everyone together for a few days is extremely valuable and productive, and your donations to KDE .e.V. help to make that possible!
Getting KMime ready for KDE FrameworksWork on moving more libraries or classes from PIM to Frameworks had been paused for some time while we were doing the transition to Qt 6. But that’s done now, so we can continue.
My main focus for this is KMime, a library for parsing and generating emails. KMime has its origins at the beginning of the century, and while for email- related code that’s actually more on the younger side it’s mature and optimized in what it does, and just needs a bit of polish to catch up with current standards.
This includes:
- Cleaning up some legacy API from a time when attachments were sort of an afterthought (to KMime but also the email standards).
- Making more of the API const-correct so parsing a message doesn’t accidentally modify it.
- Moving some code out of KMime that isn’t really in scope but just had been put there to solve dependency issues in the past.
While doing this and reviewing and adjusting consumer code we also found some duplicated, inefficient or no longer used code in libraries built on top of KMime that got subsequently cleaned up.
Moving classes to KDE FrameworksMoving code to Frameworks isn’t just about entire libraries, it can also be about a single class or method. MultiplyingLineEdit is such a case that was requested to move, and similar to KMime needed a couple of cleanups.
Sometimes it’s however also about the other way around, phasing out custom infrastructure that is meanwhile equally or better served by code in Frameworks. Pimcommon::ConfigureImmutableWidgetUtils is an example for that, with KConfigDialogManager being a more comprehensive replacement. Porting the remaining uses also progressed.
TravelTravelling to Toulouse for the sprint also provided some opportunity for field-testing Itinerary.
- The new Deutsche Bahn BahnCard “replacement documents” are recognized by Itinerary and got me through ticket checks successfully.
- For the first time I got correct automatic transfer suggestions between railway stations in Paris (ie. take the direct Metro line rather than some nonsensical long distance train chain), as well as to the venue in Toulouse (ie. recommended to walk rather than taking the Metro for one stop), thanks to Transitous. New and stricter preconditions for automatic transfers also help with improving the quality of the suggestions.
And we also got a few improvements out of this:
- Added support for creating events from OSM office elements, given the sprint was hosted in the rooms of a co-working space.
- Company capital as sometimes mentioned in the fine print of French business documents (such as hotel bookings) doesn’t confuse the generic price extractor anymore, so your hotel stay is no longer mistakenly shown to cost 3.000.000 Euro.
The return trip turned out particularly exciting with a rare railway electricity outage, which provided ideas for how to extend Itinerary’s alternative connection search.
This week in KDE: Plasma 6.1 cleanups
Plasma 6.1 has been released to good reviews! We’ve spent the week fixing issues reported so far, as always. So far we’re in good shape here, with almost all the big issues fixed already. We’re still tracking a few more, such as cases where triple buffering introduced stuttering, or random QML widgets and System Settings pages failing to launch until Qt’s QML cache folder is cleared (if you do this, please save it first and attach it to the bug report).
So this time, lets start with the bug fixes:
Bug FixesMoving the pointer right after the screen locks to make it unlock immediately no longer just breaks it entirely instead. In addition, made the screen locker more robust against failure in a few more cases (Xaver Hugl, Plasma 6.1.1. Link 1, link 2, and link 3)
Re-fixed the bug of desktop files dragged to another screen disappearing until Plasma was restarted. This so fixed a case where Plasma could crash when dragging files from the desktop to some folders in Dolphin (David Edmundson, Plasma 6.1.1. Link 1 and link 2)
In Plasma’s new zoomed-out Edit Mode, moving non-center-aligned panels to a different screen edge once again works rather than crashing Plasma (Marco Martin, Plasma 6.1.1. Link)
KWin’s Cube effect can once again be opened reliably (David Edmundson, Plasma 6.1.1. Link)
KWin’s Zoom effect and ICC color profiles now get along better (Xaver Hugl, Plasma 6.1.1. Link)
KWin’s Shake Cursor effect now works as expected with every global animation speed, including animations entirely disabled (Vlad Zahorodnii, Plasma 6.1.1. Link)
Fixed a recent visual regression KWin’s Glide effect (Vlad Zahorodnii, Plasma 6.1.1. Link)
Widgets on the Plasma desktop that are dragged while in the new zoomed-out Edit Mode are now connected to the pointer as expected (Marco Martin, Plasma 6.1.1. Link)
In Discover, the “still looking” busy indicator is no longer visually broken for the first search you make after launching the app (Akseli Lahtinen, Plasma 6.1.1. Link)
Fixed that weird blur glitch when a floating panel de-floats (Marco Martin, KDE Frameworks 6.3.1. Link)
System Settings’ Audio Volume page no longer causes System Settings to crash when you open it for a second time(David Redondo, Qt 6.7.2. Link)
When you drag pinned Task Manager icons to re-arrange them, other icons no longer come along for the ride and ruin everything (Niccolò Venerandi, Qt 6.7.2. Link)
Other bug information of note:
- 5 Very high priority Plasma bugs (same as last week). Current list of bugs
- 33 15-minute Plasma bugs (down from 37 last week). Current list of bugs
- 139 KDE bugs of all kinds fixed over the last week. Full list of bugs
You can now resize the sidebar and the playlist bar in Elisa to suit your preferences. This is enabled by the recent work to get better a resizable split view control, so expect more of these soon! (Jack Hill, Elisa 24.08.0. Link)
Removed Filelight’s back and forward buttons because they were confusing when paired with the “go up” action (Han Young, Filelight 24.08.0. Link)
A number of countries that use Metric units with the US Letter paper size (e.g. Canada) now get the correct paper size set in System Settings’ Region & Language Page (Han Young, Plasma 6.1.1. Link)
By default, Breeze-themed windows can now be dragged only from their logical and visually distinct header areas, rather than every empty area. This is particularly helpful for control-heavy apps with with lots of draggable UI elements, such as Kdenlive. You can of course change this back if you’d like (me: Nate Graham, Plasma 6.2.0. Link)
In Discover, Flatpak runtimes on the Updates page are now shown in a separate “Application Support” category to clarify their purpose (Ivan Tkachenko, Plasma 6.2.0. Link)
Plasma’s Weather widget now tries to guide you away from weather stations provided by wetter.com, which doesn’t even include current temperature data! These weather stations will now only be shown as a fallback if no other better weather stations are available (Ismael Asensio, Plasma 6.2.0. Link)
Plasma’s Bluetooth widget no longer shows devices you’ve blocked (Ivan Tkachenko, Plasma 6.2.0. Link)
In System Monitor, you can now hide a column from the context menu you’ll see when right-clicking on its header (James Graham, Plasma 6.2.0. Link)
Reverted a change made a few months ago to force single-click on Folder View popups in list mode. These are file views and not menus — despite any superficial similarity — so we now treat them like file views and respect your click preference (me: Nate Graham, Plasma 6.2.0. Link)
System Settings’ Quick Settings pages is now visible in the sidebar rather than the header, matching other KDE apps (me: Nate Graham, Plasma 6.2.0. Link)
The Get New [thing] dialogs now use a more compact view style by default, improving information density (me: Nate Graham, Frameworks 6.4. Link)
Automation & SystematizationAdded some GUI tests for various functionality in KWrite (Antoine, Herlicq link)
Added an autotest to verify screen arrangements after XWayland scales change (Xaver Hugl, link)
Added an autotest to verify that power profiles work as expected (Fushan Wen, link)
…And Everything ElseThis blog only covers the tip of the iceberg! If you’re hungry for more, check out https://planet.kde.org, where you can find more news from other KDE contributors.
How You Can HelpThe KDE organization has become important in the world, and your time and labor have helped to bring it there! But as we grow, it’s going to be equally important that this stream of labor be made sustainable, which primarily means paying for it. Right now the vast majority of KDE runs on labor not paid for by KDE e.V. (the nonprofit foundation behind KDE, of which I am a board member), and that’s a problem. We’ve taken steps to change this with paid technical contractors — but those steps are small due to growing but still limited financial resources. If you’d like to help change that, consider donating today!
Otherwise, visit https://community.kde.org/Get_Involved to discover other ways to be part of a project that really matters. Each contributor makes a huge difference in KDE; you are not a number or a cog in a machine! You don’t have to already be a programmer, either. I wasn’t when I got started. Try it, you’ll like it! We don’t bite!
Abhijith PA: abhijithpa.me to abhijithpa.in
I let go my domain abhijithpa.me. It was getting expensive and I don’t fancy that anymore. I never actually purchased this domain but came with a package offer. I then kept it for couple of years.
I have now bought a new domain abhijithpa.in and pointed everything to it.
So if you are seeing abhijithpa.me with lot of contents, that is no me. Its either domain squatters or someone impersonating me.
Promet Source: [Study] U.S. Government CMS Preferences and Trends
Russ Allbery: Review: And the Stars Will Sing
Review: And the Stars Will Sing, by Michelle Browne
Series: The Meaning Wars #1 Publisher: Michelle Browne Copyright: 2012, 2021 Printing: 2021 ASIN: B0075G7GEA Format: Kindle Pages: 85And the Stars Will Sing is a self-published science fiction novella, the first of a (currently) five book series. I believe it may be Browne's first publication, although I don't have a good data source to confirm.
Crystal Weiss is a new graduate from Mars, about to leave the solar system to her first job assignment: installation of a permanent wormhole in the vicinity of Messier 14. Her expertise is the placement calculations. The heavy mathematical lifting is of course done by computers, but humans have to do the mapping and some of the guidance. And the Stars Will Sing is an epistolary novel, told in the form of her letters to her friend Sarah.
I feel bad when I stumble across a book like this. I want to stick with my habit of writing a review of each book I read, but it's one thing to pan a bad book by a famous author and another thing to pick on a self-published novella that I read due to some recommendation or mention whose details I've forgotten. Worse, I think this wasn't even the recommended book; I looked up the author, saw that the first of a series was on sale, and thought "oh, hey, I like epistolary novels and I'm in the mood for some queer space opera."
This book didn't seem that queer (there is a secondary lesbian relationship but the main relationship seemed rather conventional), but I'll get to the romance in a moment.
I was not the reader for this book. There's a reason why most of the books I read are from traditional publishers; I'm too critical of a reader for a lot of early self-published work. It's not that I dislike self-publishing as a concept — many self-published books are excellent and the large publishers have numerous problems — but publishers enforce a quality bar. Inconsistently, unfairly, and by rejecting a lot of good work, but still, they do. I'm fairly sure traditional publishers would have passed on this book; the quality of the writing isn't there yet.
(It's certainly a better book than I could have written! But that's why I'm writing my reviews over in my quiet corner of the Internet and not selling fiction to other people.)
The early chapters aren't too bad, although they have a choppy, cliched style that more writing experience usually smoothes out. The later chapters have more dialogue, enough that I started wondering how Crystal could remember that much dialogue verbatim to put into a letter, and it's not good. All of the characters talk roughly the same (even the aliens), the dialogue felt even more cliched than the rest of the writing, and I started getting distracted by the speech tags.
Crystal comes across as very young, impulsive, and a drama magnet who likes being all up in her coworkers' business. None of these are objective flaws in the book, but I could tell early on that I was going to find her annoying. She has a heavily-foreshadowed enemies-to-lovers thing with one of her male coworkers. Her constant complaining about him at the start of the story was bad enough, but the real problem is that in the very few places where he has more personality than plastic lawn furniture, he's being obnoxious to Crystal. I'm used to being puzzled by a protagonist's choice in love interests, but this one felt less like an odd personality choice and more a lack of writing skill. Even if the relationship is being set up for failure (not true by the end of this book), you've got to help me understand what the protagonist saw in him or was getting out of the relationship.
The plot was so predictable that it ironically surprised me. I was sure that some sort of twist or complication was coming, but no. I will give Browne some credit for writing a slightly more realistic character reaction to violence than most SF authors, but there was nothing in the plot to hold my interest. The world-building was generic science fiction with aliens. It had a few glimmers of promise, but there was some sort of psychic hand-waving involved in siting wormholes that didn't work for me and the plot climax made no sense to me whatsoever.
This is the kind of bad book that I don't want to hold against the writer. Twelve years later and with numerous other novels and novellas under her belt, her writing is probably much better. I do think this book would have benefited from an editor telling her it wasn't good enough for publication yet, but that's not how the Kindle self-publishing world works. Mostly, this is my fault: I half-followed a recommendation into an area of publishing that I know from past experience I should avoid without a solid review from an equally critical reader.
Followed by The Stolen, a two-story collection.
Rating: 3 out of 10
automake @ Savannah: automake 1.16.92 pretest release candidate
automake 1.16.92 pretest release candidate released. Please test if you can, so 1.17 will be as reliable as we can make it. Announcement:
https://lists.gnu.org/archive/html/autotools-announce/2024-06/msg00001.html
DataWars.io: Replit Teams for Education Deprecation: All you need to know | DataWars
Dirk Eddelbuettel: nanotime 0.3.9 on CRAN: Bugfix
A quick bug fix release 0.3.9 for our nanotime package is now on CRAN, following up on the 0.3.8 release made this week. nanotime relies on the RcppCCTZ package (as well as the RcppDate package for additional C++ operations) and offers efficient high(er) resolution time parsing and formatting up to nanosecond resolution, using the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from a rigorous refactoring by Leonardo who not only rejigged nanotime internals in S4 but also added new S4 types for periods, intervals and durations.
The 0.3.8 release added a accurate parameter for POSIXct conversions, and it turns out that this did not test as expected on arm64 so we disabled the test on that platform. The NEWS snippet below has the full details.
Changes in version 0.3.9 (2024-06-21)Thanks to my CRANberries, there is a diffstat report for this release. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository – and all documentation is provided at the nanotime documentation site.
If you like this or other open-source work I do, you can sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
Ned Batchelder: Coverage at a crossroads
This is an interesting time for coverage.py: I’m trying to make use of new facilities in Python to drastically reduce the execution-time overhead, but it’s raising tricky questions about how coverage should work.
The current situation is a bit involved. I’ll try to explain, but this will get long, with a few sections. Come talk in Discord about anything here, or anything about coverage at all.
- How coverage works today
- SlipCover
- Python 3.12: sys.monitoring
- Using lines for branches
- Problems
- Where are we?
Much of this is discussed in How coverage.py works, but I’ll give a quick overview here to set the stage for the rest of this post.
Trace functionCoverage knows what code is executed because it registers a trace function which gets called for each line of Python execution. This is the source of the overhead: calling a function is expensive. Worse, for statement coverage, we only need one bit of information for each line: was it ever executed? But the trace function will be invoked for every execution of the line, taking time but giving us no new information.
ArcsThe other thing to know about how coverage works today is arcs. Coverage measures branch coverage by tracking the previous line that was executed, then the trace function can record an arc: the previous line number and the current line number as a pair. Taken all together, these arcs show how execution has moved through the code.
Most arcs are uninteresting. Consider this simple program:
1print("Hello")2print("world")
3print("bye")
This will result in arcs (1, 2) and (2, 3). Those tell us that lines 1, 2, and 3 were all executed, but nothing more interesting. Lots of arcs are this kind of straight-line information.
But when there are choices in the execution path, arcs tell us about branches taken:
1a = 12if a == 1:
3 print("a is one!")
4else:
5 print("a isn't one!")
6print("Done")
Now we’ll collect these arcs during execution: (1, 2), (2, 3), (3, 6). When coverage.py analyzes the code, it will determine that there were two possible arcs that didn’t happen: (2, 5) and (5, 6).
The set of all possible arcs is used to determine where the branches are. A branch is a line that has more than one possible arc leaving it. In this case, the possible arcs include (2, 3) and (2, 5), so line 2 is a branch, and only one of the possible arcs happened, so line 2 is marked as a partial branch.
SlipCoverSlipCover is a completely different implementation of a coverage measurement tool, focused on minimal execution overhead. They’ve accomplished this in a few clever ways, primarily by instrumenting the code to directly announce what is happening rather than using a trace function. Synthetic bytecode or source lines are inserted into your code to record data during execution without using a trace function.
SlipCover’s author (Juan Altmayer Pizzorno) and I have been talking for years about how SlipCover and coverage.py each work, with the ultimate goal to incorporate SlipCover-like techniques into coverage.py. SlipCover is an academic project, so was never meant to be widely used and maintained long-term.
One of the ways that SlipCover reduces overhead is to remove instrumentation once it has served its purpose. After a line has been marked as executed, there is no need to keep that line’s inserted bytecode. The extra tracking code can be removed to avoid its execution overhead.
Instrumenting and un-instrumenting code is complex. With Python 3.12, we might be able to get the best aspects of instrumented code without having to jump through complicated hoops.
Python 3.12: sys.monitoringPython 3.12 introduced a new way for Python to track execution. The sys.monitoring feature lets you register for events when lines are executed. This is similar to a classic trace function, but the crucial difference is you can disable the event line-by-line. Once a line has reported its execution to coverage.py, that line’s event can be disabled, and the line will run at full speed in the future without the overhead of reporting to coverage. Other lines in the same file will still report their events, and they can each be disabled once they have fired once. This gives us low overhead because lines mostly run at full speed.
Coverage.py has supported line coverage in 3.12 since 7.4.0 with only 5% overhead or so.
Unfortunately, branch coverage is a different story. sys.monitoring has branch events, but they are disabled based only on the “from” line, not on the “from/to” pair. In our example above, line 2 could branch to 3 or to 5. When sys.monitoring tells us line 2 branched to 3, we have to keep the event enabled until we get an event announcing line 2 branching to line 5. This means we continue to suffer event overhead during execution of the 2-to-3 branch, plus we have to do our own bookkeeping to recognize when both branch destinations have been seen before we can disable the event.
As a result, branch coverage doesn’t get the same advantages from sys.monitoring as statement coverage.
I’ve been talking to the core Python devs about changing how sys.monitoring works for branches. But even if we come to a workable design, because of Python’s yearly release cycle it wouldn’t be available until 3.14 in October 2025 at the earliest.
Using lines for branchesIn SlipCover, Juan came up with an interesting approach to use sys.monitoring line events for measuring branch coverage. SlipCover already had a few ways of instrumenting code, rewriting the code during import to add statements that report execution data. The new idea was to add lines at the destinations of branches. The lines don’t have to do anything. If sys.monitoring reports that the line was executed, then it must mean that the branch to that line happened.
As an example, our code from above would be rewritten to look something like this:
a = 1if a == 1: # line 2
NO-OP # line A, marked as 2->3
print("a is one!") # line 3
else:
NO-OP # line B, marked as 2->5
print("a isn't one!") # line 5
print("Done")
If sys.monitoring reports that line A was executed, we can record a branch from 2 to 3 and disable the event for line A. This reduces the overhead and still lets us later get an event for line B when line 2 branches to line 5.
This seems to give us the best of all the approaches: events can be disabled for each choice from a branch independently, and the inserted lines can be as minimal as possible.
ProblemsThere are a few problems with adapting this clever approach to coverage.py.
Moving away from arcsThis technique changes the data we collect during execution. We no longer get a (1, 2) arc, for example. Overall, that’s fine because that arc isn’t involved in computing branches anyway. But arcs are used as intermediate data throughout coverage.py, including its extensive test suite. How can we move to arc-less measurement on 3.12+ without refactoring many tests and while still supporting Pythons older than 3.12?
I’ve gotten a start on adapting some test helpers to avoid having to change the tests, so this might not be a big blocker.
Is every multi-arc a branch?Another problem is how coverage.py determines branches. As I mentioned above, coverage.py statically analyzes code to determine all possible arcs. Any line that could arc to more than one next line is considered a branch. This works great for classic branches like if statements. But what about this code?
1def func(x):2 try:
3 if x == 10:
4 print("early return")
5 return
6 finally:
7 print("finally")
8 print("finished")
If you look at line 7, there are two places it could go next. If x is 10, line 7 will return from the function because of the return on line 5. If x is not 10, the line 7 will be followed by line 8. Coverage.py’s static analysis understands these possibilities and includes both (7, return) and (7, 8) in the possible arcs, so it considers line 7 a branch. But is it? The conditional here is really on line 3, which is already considered a branch.
I mention this as a problem here because the clever NO-OP rewriting scheme depends on being able to insert a line at a destination that clearly indicates where the branch started from. In this finally clause, where would we put the NO-OP line for the return? The rewriting scheme breaks down if the same destination can be reached from different starting points.
But maybe those cases are exactly the ones that shouldn’t be considered branches in the first place?
Where are we?I’ve been hacking on this in a branch in the coveragepy repo. It’s a complete mess with print-debugging throughout just to see where this idea could lead.
For measuring performance, we have a cobbled-together benchmark tool in the benchmark directory of the repo. I could use help making it more repeatable and useful if you are interested.
I’m looking for thoughts about how to resolve some of these issues, and how to push forward on a faster coverage.py. There’s now a #coverage-py channel in the Python Discord where I’d welcome feedback or ideas about any of this, or anything to do with coverage.py.
Real Python: The Real Python Podcast – Episode #209: Python's Command-Line Utilities & Music Information Retrieval Tools
What are the built-in Python modules that can work as useful command-line tools? How can these tools add more functionality to Windows machines? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Web Review, Week 2024-25
Let’s go for my web review for the week 2024-25.
Proton is transitioning towards a non-profit structureTags: tech, internet, ethics, privacy
Very interesting move. I wish them well!
https://proton.me/blog/proton-non-profit-foundation
Tags: tech, java
Oracle doing Oracle things I guess… The surprising bit to me is the fact that so many people still seem to use Java SE while there are other excellent alternatives.
https://www.theregister.com/2024/06/20/oracle_java_licence_teams/
Tags: tech, microsoft, security
A deep dive into the events which led to the SolarWinds breaches. The responsibility from Microsoft as an organization is staggering. Their handling of security matters massively failed once more. I don’t get how governmental agencies or other companies can still turn to Microsoft with sensitive data.
https://www.propublica.org/article/microsoft-solarwinds-golden-saml-data-breach-russian-hackers
Tags: tech, microsoft, security
Very unsurprising, the harm is probably done though. They’ll have to work hard for their reputation to recover (even though it was probably low already).
Tags: tech, gpt, surveillance
It was already hard to trust this company, but now… that clearly gives an idea of the kind of monetization channels they’re contemplating.
https://futurism.com/the-byte/snowden-openai-calculated-betrayal
Tags: tech, ai, machine-learning, gpt, copilot, security, privacy
The creative ways to exfiltrate data from chat systems built with LLMs…
https://embracethered.com/blog/posts/2024/github-copilot-chat-prompt-injection-data-exfiltration/
Tags: tech, ai, machine-learning, gpt, data-science, criticism, funny
OK, this is a rant about the state of the market and people drinking kool-aid. A bit long but I found it funny and well deserved at times.
https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you-if-you-mention-ai-again/
Tags: tech, ai, machine-learning, gpt, self-hosting, criticism
Since there are ways to offset the plagiarism a bit, let’s do it. Obviously it’s not perfect but that’s a start.
https://blog.zgp.org/block-ai-training-on-a-web-site/
Tags: tech, foss, maintenance, life, history
Very interesting piece… shows how someone can end up maintaining something essential for decades. This is a lesson for us all.
https://lwn.net/SubscriberLink/978463/be23210c163a2107/
Tags: tech, networking, security, self-hosting, internet
This is indeed a real concern… with no propre solution in sight.
https://www.macchaffee.com/blog/2024/ddos-attacks/
Tags: tech, networking, security
On the peculiarities of running a network for a university… this is an interesting way to frame it as basically being an ISP with benefits.
https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OurNetworkTrafficIsUnknown
Tags: tech, shell, scripting
This is indeed an easy mistake to do. It’s better be avoided.
https://mywiki.wooledge.org/ParsingLs
Tags: tech, tools, git, cad
Interesting trick for a zip based format containing mostly text.
https://blog.lambda.cx/posts/freecad-and-git/
Tags: tech, python, memory, performance
Interesting dive into how join() and generator behave in CPython.
https://berglyd.net/blog/2024/06/joining-strings-in-python/
Tags: tech, programming, python
That’s what happens where references are half hidden in a language. You think each closure get a different copy but in fact they all refer to the same object.
https://utcc.utoronto.ca/~cks/space/blog/python/UnderstandingClosureOddity
Tags: tech, json, security
JSON, its grammar and the security implications. The approach of looking at a restricted subset is interesting.
https://neilmadden.blog/2023/05/31/regular-json/
Tags: tech, programming, rust
Ever wondered how this operator is implemented in Rust? It’s not that complicated.
https://blog.sulami.xyz/posts/demystifying-rusts-questionmark-operator/
Tags: tech, data-visualization
Why box plots are hard to grasp and probably badly designed. There are good alternatives out there though.
https://nightingaledvs.com/ive-stopped-using-box-plots-should-you/
Tags: tech, complexity, probability, simulation
Some problems are indeed tackled faster by having a simulation allowing to explore potential solutions. It’s tempting to go very formal and theoretical but it’d require more effort and be more error prone.
https://sirupsen.com/napkin/problem-16-simulation
Tags: tech, library, api, maintenance
Good musing about major version numbers and backward compatibility. It is indeed important to communicate breaking changes properly and to not have those too often.
https://blog.cessen.com/post/2022_07_09_major_version_numbers_may_not_be_sacred
Tags: tech, software, programming, work, complexity
It might not look like a lot from the outside, but “just implementation details” in fact hides quite some work and complexity.
https://ntietz.com/blog/whats-behind-just-implementation/
Tags: tech, software, organization, complexity
Very nice piece about the various types of complexities we encounter in our trade, and what we can or should do about it.
https://olano.dev/blog/a-note-on-essential-complexity
Tags: tech, software, management
This is a funny pretense, and yet… If any of this remind you of a real context, this would be paper cuts. Have enough of those and indeed the organization might grind to a halt.
https://erikbern.com/2023/12/13/simple-sabotage-for-software.html
Tags: tech, requirements, software, product-management
This is indeed a good way to classify events probability in requirements. It definitely impacts how you handle them in software.
https://lukeplant.me.uk/blog/posts/never-sometimes-always/
Tags: tech, communication, talk
Nice trick, definitely should use it more often.
https://tidyfirst.substack.com/p/start-presentations-on-the-second
Tags: tech, information, social-media, criticism
Indeed the analogy from “ultra-processed food” is an interesting one in the information context.
https://calnewport.com/on-ultra-processed-content/
Bye for now!
mark.ie: My Drupal Core Contributions for week-ending June 21th, 2024
Here's what I've been working on for my Drupal contributions this week. Thanks to Code Enigma for sponsoring the time to work on these.
Bits from Debian: Looking for the artwork for Trixie the next Debian release
Each release of Debian has a shiny new theme, which is visible on the boot screen, the login screen and, most prominently, on the desktop wallpaper. Debian plans to release Trixie, the next release, next year. As ever, we need your help in creating its theme! You have the opportunity to design a theme that will inspire thousands of people while working in their Debian systems.
For the most up to date details, please refer to the wiki.
We would also like to take this opportunity to thank Juliette Taka Belin for doing the Emerald theme for bookworm.
The deadlines for submissions is: 2024-09-19
The artwork is usually picked based on which themes look the most:
- ''Debian'': admittedly not the most defined concept, since everyone has their own take on what Debian means to them.
- ''plausible to integrate without patching core software'': as much as we love some of the insanely hot looking themes, some would require heavy GTK+ theming and patching GDM/GNOME.
- ''clean / well designed'': without becoming something that gets annoying to look at a year down the road. Examples of good themes include Joy, Lines, Softwaves and futurePrototype.
If you'd like more information or details, please post to the Debian Desktop mailing list.
Bits from Debian: Recherche d'un thème pour Trixie, la prochaine version de Debian
Chaque édition de Debian bénéficie d'un nouveau thème brillant visible sur l'écran d'amorçage, l'écran de connexion et, de la façon la plus évidente, sur le fond d'écran. Debian prévoit de publier Trixie, sa prochaine version, durant l'année à venir. Et, comme toujours, nous avons besoin de vous pour créer son thème ! Vous avez l'occasion de concevoir un thème qui inspirera des milliers de personnes quand ils travaillent sur leur machine Debian.
Pour disposer des détails les plus récents, veuillez vous référer à la page du wiki.
Nous voudrions profiter de cette occasion pour remercier Juliette Taka Belin pour avoir créé le thème Emerald pour Bookworm.
La date limite pour les propositions est le 19 septembre 2024.
Le thème est habituellement choisi en se basant sur ce qui paraît le plus :
- « Debian » : il est vrai que ce n'est pas le concept le mieux défini, dans la mesure où chacun a son sentiment sur ce que représente Debian pour eux ;
- « possible à intégrer sans corriger le logiciel de base » : autant nous aimons les thèmes follement excitants, certains nécessiteraient un gros travail d'adaptation des thèmes GTK+ et la correction de GDM/GNOME ;
- « clair et bien conçu » : sans être quelque chose qui devient ennuyeux à regarder au bout d'un an. Parmi les bons thèmes, on peut citer Joy, Lines, Softwaves et futurePrototype.
Si vous souhaitez disposer plus d'informations, veuillez utiliser la liste de diffusion Debian Desktop.
Bits from Debian: Looking for the artwork for Trixie the next Debian release
Each release of Debian has a shiny new theme, which is visible on the boot screen, the login screen and, most prominently, on the desktop wallpaper. Debian plans to next release, Trixie, next year. As ever, we need your help in creating its theme! You have the opportunity to design a theme that will inspire thousands of people while working in their Debian systems.
For the most up to date details, please refer to the wiki.
We would also like to take this opportunity to thank Juliette Taka Belin for doing the Emerald theme for bookworm.
The deadlines for submissions is: 2024-09-19
The artwork is usually picked based on which themes look the most:
- ''Debian'': admittedly not the most defined concept, since everyone has their own take on what Debian means to them.
- ''plausible to integrate without patching core software'': as much as we love some of the insanely hot looking themes, some would require heavy GTK+ theming and patching GDM/GNOME.
- ''clean / well designed'': without becoming something that gets annoying to look at a year down the road. Examples of good themes include Joy, Lines, Softwaves and futurePrototype.
If you'd like more information or details, please post to the Debian Desktop mailing list.
mark.ie: A bash script to set up Drupal for local development using DDEV
Last week I wrote about how to set up Drupal for core contributing using DDEV. This week I decided to write a bash script so I wouldn't have to remember what I did, it would "just work".
health @ Savannah: MyGNUHealth 2.2 series released!
Dear all
I am happy to announce the release of MyGNUHealth 2.2.0!
The new series of the GNU Health Personal Health record comes with many improvements and bug fixes. Some highlights of this new version:
- Support for Kivy 2.3.0
- Localization. MyGNUHealth now has support for different languages. English, Spanish and Chinese are available to use, and French, German, Italian are ready to be translated. There will be a translation component for MyGNUHealth at Codeberg's Weblate instance.
- Bluetooth functionality: Starting with MyGH series 2.2 we provide bluetooth integration for open compatible devices and health trackers. We include the link with the Pinetime Smartwatch (experimental) and the possibility to link to any open hardware device (glucometer, scales, blood pressure monitors, .. ). We need to get a list of available medical devices that respect our privacy and freedom, so let us know of any!
- Charts now allow to select date ranges with calendar widgets
- The Book of Life have a revised format for the pages.
- The charts have been improved in the format and include x axis labels.
Thanks to Kivy, Mygnuhealth codebase can be ported to other architectures and operating systems such as Android AOSP (Pierre Michel is working on this) and GNU/Linux phones.
In addition to Savannah, we have incorporated Codeberg to the GNU Health development environment. Mailing lists, news and file downloads are at GNU, while the development repositories are at Codeberg (https://codeberg.org/gnuhealth)
You can download the latest MyGNUhealth sourcecode from GNU ftp site, pypi (using pip) or from your operating system package (like openSUSE).
Upgrading should be straightforward, and all the health history will remain in the MyGH database. In any case, please make sure you make a backup before upgrading (and daily ;) ).
Thank you to all the contributors that have possible this milestone!
Happy hacking
Luis
death and gravity: reader 3.13 released – scheduled updates
Hi there!
I'm happy to announce version 3.13 of reader, a Python feed reader library.
What's new? #Here are the highlights since reader 3.12.
Scheduled updates #reader now allows updating feeds at different rates via scheduled updates.
The way it works is quite simple: each feed has an update interval that determines when the feed should be updated next; calling update_feeds(scheduled=True) updates only feeds that should be updated at or before the current time.
The interval can be configured by the user globally or per-feed through the .reader.update tag. In addition, you can specify a jitter; for an interval of 24 hours, a jitter of 0.25 means the update will occur any time in the first 6 hours of the interval.
In the future, the same mechanism will be used to handle 429 Too Many Requests.
Improved documentation #As part of rewriting the Updating feeds user guide section to talk about scheduled updates, I've added a new section about being polite to servers.
Also, we have a new recipe for adding custom headers when retrieving feeds.
mark_as_read reruns #You can now re-run the mark_as_read plugin for existing entries by adding the .reader.mark-as-read.once tag to a feed. Thanks to Michael Han for the pull request!
That's it for now. For more details, see the full changelog.
Want to contribute? Check out the docs and the roadmap.
Learned something new today? Share this with others, it really helps! PyCoder's Weekly HN Reddit linkedin Twitter
What is reader? #reader takes care of the core functionality required by a feed reader, so you can focus on what makes yours different.
reader allows you to:
- retrieve, store, and manage Atom, RSS, and JSON feeds
- mark articles as read or important
- add arbitrary tags/metadata to feeds and articles
- filter feeds and articles
- full-text search articles
- get statistics on feed and user activity
- write plugins to extend its functionality
...all these with:
- a stable, clearly documented API
- excellent test coverage
- fully typed Python
To find out more, check out the GitHub repo and the docs, or give the tutorial a try.
Why use a feed reader library? #Have you been unhappy with existing feed readers and wanted to make your own, but:
- never knew where to start?
- it seemed like too much work?
- you don't like writing backend code?
Are you already working with feedparser, but:
- want an easier way to store, filter, sort and search feeds and entries?
- want to get back type-annotated objects instead of dicts?
- want to restrict or deny file-system access?
- want to change the way feeds are retrieved by using Requests?
- want to also support JSON Feed?
- want to support custom information sources?
... while still supporting all the feed types feedparser does?
If you answered yes to any of the above, reader can help.
The reader philosophy #- reader is a library
- reader is for the long term
- reader is extensible
- reader is stable (within reason)
- reader is simple to use; API matters
- reader features work well together
- reader is tested
- reader is documented
- reader has minimal dependencies
So you can:
- have full control over your data
- control what features it has or doesn't have
- decide how much you pay for it
- make sure it doesn't get closed while you're still using it
- really, it's easier than you think
Obviously, this may not be your cup of tea, but if it is, reader can help.