Feeds
Daniel Roy Greenfeld: TIL: types.SimpleNamespace is a Bunch class
LostCarPark Drupal Blog: Drupal Advent Calendar 2024 Addendum - The Drop Times and 24 Days of Automation
I hope you enjoyed this year’s Drupal Advent Calendar, and our whistle-stop tour of Starshot tracks. If you missed it, you can catch up here.
I want to touch on a couple of things that I missed during the regular calendar schedule.
The Drop TimesThe Drop Times is a Drupal news site that covers everything happening across the Drupalverse. They do a superb job of covering DrupalCons and Drupal Camps around the world. Since the announcement of the Starshot initiative, they have been dutyfully covering all the news from the initiative, and they have been a valuable source of information when…
FSF Events: Free Software Directory meeting on IRC: Friday, December 27, starting at 12:00 EST (17:00 UTC)
PyCoder’s Weekly: Issue #661 (Dec. 24, 2024)
#661 – DECEMBER 24, 2024
View in Browser »
What are the current approaches for analyzing emotions within a piece of text? Which tools and Python packages should you use for sentiment analysis? This week, Jodie Burchell, developer advocate for data science at JetBrains, returns to the show to discuss modern sentiment analysis in Python.
REAL PYTHON podcast
A Directed Acyclic Graph (DAG) is a common data structure used to contain a series of related items that must have certain order or dependency. Topological sorting is used to help find where you might start processing to get in order handling of the items in a DAG.
REDOWAN DELOWAR
This series explores the critical security principles every Python web developer needs. The first post delves into fundamental security best practices, ranging from general principles to specific Python-related techniques.
MICHAEL FORD
In this quiz, you’ll test your understanding of removing items from lists in Python. This is a fundamental skill in Python programming, and mastering it will enable you to manipulate lists effectively.
REAL PYTHON
In this in-depth video course, you’ll learn how to build a socket server and client with Python. By the end, you’ll understand how to use the main functions and methods in Python’s socket module to write your own networked client-server applications.
REAL PYTHON course
Python decorators are one of Hashim’s favorite features. This post covers some examples he’s used in his projects. It includes the Prometheus Histogram Timing Decorators and OpenTelemetry (OTel) Manual Span Decorators.
HASHIM COLOMBOWALA
“The pipeline design pattern (also known as Chain of Command pattern) is a flexible way to handle a sequence of actions, where each handler in the chain processes the input and passes it to the next handler.”
JUAN JOSÉ EXPÓSITO GONZÁLEZ
The later in your development process that you discover the bug the more expensive it is. Shift-Left Testing is a collection of techniques to attempt to move bug discovery earlier in your process.
ANTONELLO ZANINI
There are multiple ways of merging two or more dictionaries in Python. This post teaches you how to do it and how to deal with corner cases like duplicate keys.
TREY HUNNER
Adam runs a quiz on Django at his Django London meetup. He’s shared it so you can try it yourself. Test how much you know about your favorite web framework.
ADAM JOHNSON
This post compares many of the different web frameworks available for Python. It covers: Reflex, Django, Flask, Gradio, Streamlit, Dash, and FastAPI.
TOM GOTSMAN
The post contains an embedded JupyterLite notebook containing a cookbook for SQLAlchemy. It focuses on the patterns you use in everyday ORM coding.
JAMIE CHANG
Here’s a technique for using pdb within Django through hooking specific SQL queries. This uses database instrumentation in the Django ORM.
ADAM JOHNSON
GITHUB.COM/ROCKHOPPER-TECHNOLOGIES
statsmodels: Statistical Modeling and Econometrics in Python htmy: Async, Pure-Python Rendering Engine Events SPb Python Drinkup December 26, 2024
MEETUP.COM
December 28, 2024
MEETUP.COM
December 28, 2024
PYTHON.ORG.BR
December 31, 2024
GOOGLE.COM
January 2, 2025
MEETUP.COM
Happy Pythoning!
This was PyCoder’s Weekly Issue #661.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
Divine Attah-Ohiemi: Seamless Transitions: Mastering Apache Redirects for a Smooth Hugo Migration
This week, I dove into setting up redirects with Apache to make the transition to Hugo's multilingual system smoother. The challenge? Ensuring that all those old links still worked while I migrated to the new URL format.
For instance, I needed to redirect:
/es/distrib to /distrib/index.es.html
/es/social_contract to /social_contract.es.html
/es/intro/about to /intro/about.es.html
/da to /index.da.html
To tackle this, I turned to Apache's mod_rewrite. Here’s the magic I came up with in my .htaccess file:
RewriteCond %{REQUEST_URI} ^/([a-z]{2}(?:-[a-z]{2})?)/(.*)$
RewriteCond %{DOCUMENT_ROOT}/$2/index.%1.html -f
RewriteCond %{DOCUMENT_ROOT}/$1/$2 !-d
RewriteRule ^/([a-z]{2}(?:-[a-z]{2})?)/(.*)$ /$2/index.%1.html [last,redirect]
RewriteCond %{REQUEST_URI} ^/([a-z]{2}(?:-[a-z]{2})?)/(.*)$
RewriteCond %{DOCUMENT_ROOT}/$2.%1.html -f
RewriteCond %{DOCUMENT_ROOT}/$1/$2 !-d
RewriteRule ^/([a-z]{2}(?:-[a-z]{2})?)/(.*)$ /$2.%1.html [last,redirect]
What’s happening here? The rules check if the URL starts with a language code (like /es or /da). Then, they verify whether the corresponding HTML file exists. If it does, and the path isn’t a directory, voilà! The user gets redirected to the new format.
It’s a bit of a dance with conditions and rules, but it’s satisfying to see everything working seamlessly. Now, as I continue migrating content, users clicking on old links won’t end up in a digital dead end. It’s all about keeping the flow smooth and maintaining that user experience.
So, if you’re also juggling multilingual pages and thinking about making the switch to Hugo, don’t underestimate the power of mod_rewrite. It’s your best friend in the world of redirects! Happy coding!
Freelock Blog: Automatically Geolocate Santa
We've reached the last day of the calendar, and it's time for Santa's visit! Santa has been visiting some famous places all month. With the Geocoder module, Leaflet, a Geofield, and an Address field, you can automatically put each address Santa has visited on the map!
You do need to configure a geocoder source. We're using OpenStreetmap, from the geocoder-php/nominatim-provider.
Django Weblog: Welcome to our new Django accessibility team members - Eli, Marijke, Saptak, Tushar
Sarah Abderemane, Thibaud Colas and Tom Carrick are pleased to introduce four new members in the Django Accessibility team ❤️.
Marijke Luttekes
Eliana Rosselli
Tushar Gupta
Saptak Sengupta
Marijke (pronounced Mah-Rye-Kuh) is a freelance web developer who creates human-friendly applications. She is based in Groningen, The Netherlands, specializing in Django, Python, HTML, SCSS, and vanilla JavaScript. She helps companies expand their existing projects, think about new features, train new developers, and improve developer team workflows. She is aDdjango contributor from the first session of Djangonaut Space program and she loves tea. You can learn more about Marijke on her website.
Eli is a full-stack developer from Uruguay who loves using Django and React. She is a Django contributor from the first session of the Djangonaut Space program
. She is passionate about good quality code, unit testing, and web accessibility. She enjoys drinking Maté (and talking about it!) and watching her football team play.Tushar is a software engineer at Canonical, based in India. He got involved on open source during his studies loving the the supportive community. Through fellowships like Major League Hacking, Tushar dove into Django and took part in Djangonaut Space. Learn more about Tushar on his personal website.
Saptak is a self-proclaimed Human Rights Centered Developer Web. He focuses on security, privacy, accessibility, localization, and other human rights associated with websites that makes websites more inclusive and usable by everyone. Learn more about Saptak on his personal website.
Listen to them talking about their work¶Here are recent talks or podcasts from our new team members if you want to get to know them better.
Marijke Luttekes - My path to becoming a Django contributor | pyGrunn 2024 Eli Rosselli - My step-by-step guide to becoming a Django core contributor | DjangoCon Europe 2024 Tushar Gupta - Djangonauts, Ready for Blast-Off | Talk Python to Me Ep.451 Saptak S - Accessibility for the Django Community | DjangoCon Europe 2024 What’s next¶In truth, our four new accessibility team members joined the team months ago – shortly after we published our 2023 accessibility team report. Up next, a lot of the team will be present at FOSDEM 2025, organizing, volunteering, or speaking at the Inclusive Web Devroom.
The Drop Times: Inside the Media Management Track of Drupal CMS
Specbee: Functions and filters to get you started with Twig Tweak in Drupal 10 (with examples)
LostCarPark Drupal Blog: Drupal Advent Calendar day 24 - Experience Builder
Welcome back for the final door of the 2024 Advent Calendar. We’ve already covered all 22 tracks of the Starshot initiative, as well as some non-track aspects. For our final door, we are looking at something that is absent from the initial release of Drupal CMS, but is hoped to come to fruition in 2025 and revolutionise website theming. Let me introduce Lauri Timmanee, who is here to tell us about Experience Builder.
What is Experience Builder?At DrupalCon Lille 2023, Dries announced a new strategic initiative to build a Next Generation Page Builder. The goal of the initiative was to improve…
TagsTalk Python to Me: #490: Django Ninja
LN Webworks: 8 Common Questions About Migrating from Drupal 7 to Drupal 10
Migrating from Drupal 7 to Drupal 10 is a significant upgrade for your website. While the process can be complex, even for experienced Drupal developers, it’s essential to make informed decisions along the way. As a reliable Drupal development company, we understand that a successful migration requires careful planning and execution. In this guide, we’ll answer the most common questions about Drupal migration to help you navigate this transition smoothly.
In this blog, we will shed light on some of the “what” and “how” of Drupal 7 to 10 Migration doubts and provide solutions from our experts. So, let’s get down to the most common Drupal FAQs.
Russ Allbery: Review: Number Go Up
Review: Number Go Up, by Zeke Faux
Publisher: Crown Currency Copyright: 2023 Printing: 2024 ISBN: 0-593-44382-9 Format: Kindle Pages: 373Number Go Up is a cross between a history and a first-person account of investigative journalism around the cryptocurrency bubble and subsequent collapse in 2022. The edition I read has an afterward from June 2024 that brings the story up to date with Sam Bankman-Fried's trial and a few other events. Zeke Faux is a reporter for Bloomberg News and a fellow of New America.
Last year, I read Michael Lewis's Going Infinite, a somewhat-sympathetic book-length profile of Sam Bankman-Fried that made a lot of people angry. One of the common refrains at the time was that people should read Number Go Up instead, and since I'm happy to read more about the absurdities of the cryptocurrency world, I finally got around to reading the other big crypto book of 2023.
This is a good book, with some caveats that I am about to explain at absurd length. If you want a skeptical history of the cryptocurrency bubble, you should read it. People who think that it's somehow in competition with Michael Lewis's book or who think the two books disagree (including Faux himself) have profoundly missed the point of Going Infinite. I agree with Matt Levine: Both of these books are worth your time if this is the sort of thing you like reading about. But (much) more on Faux's disagreements with Lewis later.
The frame of Number Go Up is Faux's quixotic quest to prove that Tether is a fraud. To review this book, I therefore need to briefly explain what Tether is. This is only the first of many extended digressions.
One natural way to buy cryptocurrency would be to follow the same pattern as a stock brokerage account. You would deposit some amount of money into the account (or connect the brokerage account to your bank account), and then exchange money for cryptocurrency or vice versa, using bank transfers to put money in or take it out. However, there are several problems with this. One is that swapping cryptocurrency for money is awkward and sometimes expensive. Another is that holding people's investment money for them is usually highly regulated, partly for customer safety but also to prevent money laundering. These are often called KYC laws (Know Your Customer), and the regulation-hostile world of cryptocurrency didn't want to comply with them.
Tether is a stablecoin, which means that the company behind Tether attempts to guarantee that one Tether is always worth exactly one US dollar. It is not a speculative investment like Bitcoin; it's a cryptocurrency substitute for dollars. People exchange dollars for Tether to get their money into the system and then settle all of their subsequent trades in Tether, only converting the Tether back to dollars when they want to take their money out of cryptocurrency entirely. In essence, Tether functions like the cash reserve in a brokerage account: Your Tether holdings are supposedly guaranteed to be equivalent to US dollars, you can withdraw them at any time, and because you can do so, you don't bother, instead leaving your money in the reserve account while you contemplate what new coin you want to buy.
As with a bank, this system rests on the assurance that one can always exchange one Tether for one US dollar. The instant people stop believing this is true, people will scramble to get their money out of Tether, creating the equivalent of a bank run. Since Tether is not a regulated bank or broker and has no deposit insurance or strong legal protections, the primary defense against a run on Tether is Tether's promise that they hold enough liquid assets to be able to hand out dollars to everyone who wants to redeem Tether. (A secondary defense that I wish Faux had mentioned is that Tether limits redemptions to registered accounts redeeming more than $100,000, which is a tiny fraction of the people who hold Tether, but for most purposes this doesn't matter because that promise is sufficient to maintain the peg with the dollar.)
Faux's firmly-held belief throughout this book is that Tether is lying. He believes they do not have enough money to redeem all existing Tether coins, and that rather than backing every coin with very safe liquid assets, they are using the dollars deposited in the system to make illiquid and risky investments.
Faux never finds the evidence that he's looking for, which makes this narrative choice feel strange. His theory was tested when there was a run on Tether following the collapse of the Terra stablecoin. Tether passed without apparent difficulty, redeeming $16B or about 20% of the outstanding Tether coins. This doesn't mean Faux is wrong; being able to redeem 20% of the outstanding tokens is very different from being able to redeem 100%, and Tether has been fined for lying about its reserves. But Tether is clearly more stable than Faux thought it was, which makes the main narrative of the book weirdly unsatisfying. If he admitted he might be wrong, I would give him credit for showing his work even if it didn't lead where he expected, but instead he pivots to focusing on Tether's role in money laundering without acknowledging that his original theory took a serious blow.
In Faux's pursuit of Tether, he wanders through most of the other elements of the cryptocurrency bubble, and that's the strength of this book. Rather than write Number Go Up as a traditional history, Faux chooses to closely follow his own thought processes and curiosity. This has the advantage of giving Faux an easy and natural narrative, something that non-fiction books of this type can struggle with, and it lets Faux show how confusing and off-putting the cryptocurrency world is to an outsider.
The best parts of this book were the parts unrelated to Tether. Faux provides an excellent summary of the Axie Infinity speculative bubble and even traveled to the Philippines to interview people who were directly affected. He then wandered through the bizarre world of NFTs, and his first-hand account of purchasing one (specifically a Mutant Ape) to get entrance to a party (which sounded like a miserable experience I would pay money to get out of) really drives home how sketchy and weird cryptocurrency-related software and markets can be. He also went to El Salvador to talk to people directly about the country's supposed embrace of Bitcoin, and there's no substitute for that type of reporting to show how exaggerated and dishonest the claims of cryptocurrency adoption are.
The disadvantage of this personal focus on Faux himself is that it sometimes feels tedious or sensationalized. I was much less interested in his unsuccessful attempts to interview the founder of Tether than Faux was, and while the digression into forced labor compounds in Cambodia devoted to pig butchering scams was informative (and horrific), I think Faux leaned too heavily on an indirect link to Tether. His argument is that cryptocurrency enables a type of money laundering that is particularly well-suited to supporting scams, but both scams and this type of economic slavery existed before cryptocurrency and will exist afterwards. He did not make a very strong case that Tether was uniquely valuable as a money laundering service, as opposed to a currently useful tool that would be replaced with some other tool should it go away.
This part of the book is essentially an argument that money laundering is bad because it enables crime, and sure, to an extent I agree. But if you're going to put this much emphasis on the evils of money laundering, I think you need to at least acknowledge that many people outside the United States do not want to give US government, which is often openly hostile to them, veto power over their financial transactions. Faux does not.
The other big complaint I have with this book, and with a lot of other reporting on cryptocurrency, is that Faux is sloppy with the term "Ponzi scheme." This is going to sound like nit-picking, but I think this sloppiness matters because it may obscure an ongoing a shift in cryptocurrency markets.
A Ponzi scheme is not any speculative bubble. It is a very specific type of fraud in which investors are promised improbably high returns at very low risk and with safe principal. These returns are paid out, not via investment in some underlying enterprise, but by taking the money from new investments and paying it to earlier investors. Ponzi schemes are doomed because satisfying their promises requires a constantly increasing flow of new investors. Since the population of the world is finite, all Ponzi schemes are mathematically guaranteed to eventually fail, often in a sudden death spiral of ever-increasing promises to lure new investors when the investment stream starts to dry up.
There are some Ponzi schemes in cryptocurrency, but most practices that are called Ponzi schemes are not. For example, Faux calls Axie Infinity a Ponzi scheme, but it was missing the critical elements of promised safe returns and fraudulently paying returns from the investments of later investors. It was simply a speculative bubble that people bought into on the assumption that its price would increase, and like any speculative bubble those who sold before the peak made money at the expense of those who bought at the peak.
The reason why this matters is that Ponzi schemes are a self-correcting problem. One can decry the damage caused when they collapse, but one can also feel the reassuring certainty that they will inevitably collapse and prove the skeptics correct. The same is not true of speculative assets in general. You may think that the lack of an underlying economic justification for prices means that a speculative bubble is guaranteed to collapse eventually, but in the famous words of Gary Schilling, "markets can remain irrational a lot longer than you and I can remain solvent."
One of the people Faux interviews explains this distinction to him directly:
Rong explained that in a true Ponzi scheme, the organizer would have to handle the "fraud money." Instead, he gave the sneakers away and then only took a small cut of each trade. "The users are trading between each other. They are not going through me, right?" Rong said. Essentially, he was arguing that by downloading the Stepn app and walking to earn tokens, crypto bros were Ponzi'ing themselves.
Faux is openly contemptuous of this response, but it is technically correct. Stepn is not a Ponzi scheme; it's a speculative bubble. There are no guaranteed returns being paid out of later investments and no promise that your principal is safe. People are buying in at price that you may consider irrational, but Stepn never promised you would get your money back, let alone make a profit, and therefore it doesn't have the exponential progression of a Ponzi scheme. One can argue that this is a distinction without a moral difference, and personally I would agree, but it matters immensely if one is trying to analyze the future of cryptocurrencies.
Schemes as transparently unstable as Stepn (which gives you coins for exercise and then tries to claim those coins have value through some vigorous hand-waving) are nearly as certain as Ponzi schemes to eventually collapse. But it's also possible to create a stable business around allowing large numbers of people to regularly lose money to small numbers of sophisticated players who are collecting all of the winnings. It's called a poker room at a casino, and no one thinks poker rooms are Ponzi schemes or are doomed to collapse, even though nearly everyone who plays poker will lose money.
This is the part of the story that I think Faux largely missed, and which Michael Lewis highlights in Going Infinite. FTX was a legitimate business that made money (a lot of money) off of trading fees, in much the same way that a casino makes money off of poker rooms. Lots of people want to bet on cryptocurrencies, similar to how lots of people want to play poker. Some of those people will win; most of those people will lose. The casino doesn't care. Its profit comes from taking a little bit of each pot, regardless of who wins. Bankman-Fried also speculated with customer funds, and therefore FTX collapsed, but there is no inherent reason why the core exchange business cannot be stable if people continue to want to speculate in cryptocurrencies. Perhaps people will get tired of this method of gambling, but poker has been going strong for 200 years.
It's also important to note that although trading fees are the most obvious way to be a profitable cryptocurrency casino, they're not the only way. Wall Street firms specialize in finding creative ways to take a cut of every financial transaction, and many of those methods are more sophisticated than fees. They are so good at this that buying and selling stock through trading apps like Robinhood is free. The money to run the brokerage platform comes from companies that are delighted to pay for the opportunity to handle stock trades by day traders with a phone app. This is not, as some conspiracy theories would have you believe, due to some sort of fraudulent price manipulation. It is because the average person with a Robinhood phone app is sufficiently unsophisticated that companies that have invested in complex financial modeling will make a steady profit taking the other side of their trades, mostly because of the spread (the difference between offered buy and sell prices).
Faux is so caught up in looking for Ponzi schemes and fraud that I think he misses this aspect of cryptocurrency's transformation. Wall Street trading firms aren't piling into cryptocurrency because they want to do securities fraud. They're entering this market because there seems to be persistent demand for this form of gambling, cryptocurrency markets reward complex financial engineering, and running a legal casino is a profitable business model.
Michael Lewis appears as a character in this book, and Faux portrays him quite negatively. The root of this animosity appears to stem from a cryptocurrency conference in the Bahamas that Faux attended. Lewis interviewed Bankman-Fried on stage, and, from Faux's account, his questions were fawning and he praised cryptocurrencies in ways that Faux is certain he knew were untrue. From that point on, Faux treats Lewis as an apologist for the cryptocurrency industry and for Sam Bankman-Fried specifically.
I think this is a legitimate criticism of Lewis's methods of getting close to the people he wants to write about, but I think Faux also makes the common mistake of assuming Lewis is a muckraking reporter like himself. This has never been what Lewis is interested in. He writes about people he finds interesting and that he thinks a reader will also find interesting. One can legitimately accuse him of being credulous, but that's partly because he's not even trying to do the same thing Faux is doing. He's not trying to judge; he's trying to understand.
This shows when it comes to the parts of this book about Sam Bankman-Fried. Faux's default assumption is that everyone involved in cryptocurrency is knowingly doing fraud, and a lot of his research is looking for evidence to support the conclusion he had already reached. I don't think there's anything inherently wrong with that approach: Faux is largely, although not entirely, correct, and this type of hostile journalism is incredibly valuable for society at large. Upton Sinclair didn't start writing The Jungle with an open mind about the meat-packing industry. But where Faux and Lewis disagree on Bankman-Fried's motivations and intentions, I think Lewis has the much stronger argument.
Faux's position is that Bankman-Fried always intended to steal people's money through fraud, perhaps to fund his effective altruism donations, and his protestations that he made mistakes and misplaced funds are obvious lies. This is an appealing narrative if one is looking for a simple villain, but Faux's evidence in support of this is weak. He mostly argues through stereotype: Bankman-Fried was a physics major and a Jane Street trader and therefore could not possibly be the type of person to misplace large amounts of money or miscalculate risk.
If he wants to understand how that could be possible, he could read Going Infinite? I find it completely credible that someone with what appears to be uncontrolled, severe ADHD could be adept at trading and calculating probabilities and yet also misplace millions of dollars of assets because he wasn't thinking about them and therefore they stopped existing.
Lewis made a lot of people angry by being somewhat sympathetic to someone few people wanted to be sympathetic towards, but Faux (and many others) are also misrepresenting his position. Lewis agrees that Bankman-Fried intentionally intermingled customer funds with his hedge fund and agrees that he lied about doing this. His only contention is that Bankman-Fried didn't do this to steal the money; instead, he invested customer money in risky bets that he thought would pay off. In support of this, Lewis made a prediction that was widely scoffed at, namely that much less of FTX's money was missing than was claimed, and that likely most or all of it would be found.
And, well, Lewis was basically correct? The FTX bankruptcy is now expected to recover considerably more than the amount of money owed to creditors. Faux argues that this is only because the bankruptcy clawed back assets and cryptocurrencies have gone up considerably since the FTX bankruptcy, and therefore that the lost money was just replaced by unexpected windfall profits on other investments, but I don't think this point is as strong as he thinks it is. Bankman-Fried lost money on some of what he did with customer funds, made money on other things, and if he'd been able to freeze withdrawals for the year that the bankruptcy froze them, it does appear most of the money would have been recoverable. This does not make what he did legal or morally right, but no one is arguing that, only that he didn't intentionally steal money for his own personal gain or for effective altruism donations. And on that point, I don't think Faux is giving Lewis's argument enough credit.
I have a lot of complaints about this book because I know way too much about this topic than anyone should probably know. I think Faux missed the plot in a couple of places, and I wish someone would write a book about where cryptocurrency markets are currently going. (Matt Levine's Money Stuff newsletter is quite good, but it's about all sorts of things other than cryptocurrency and isn't designed to tell a coherent story.) But if you know less about cryptocurrency and just want to hear the details of the run-up to the 2022 bubble, this is a great book for that. Faux is writing for people who are already skeptical and is not going to convince people who are cryptocurrency true believers, but that's fine. The details are largely correct (and extensively footnoted) and will satisfy most people's curiosity.
Lewis's Going Infinite is a better book, though. It's not the same type of book at all, and it will not give you the broader overview of the cryptocurrency world. But if you're curious about what was going through the head of someone at the center of all of this chaos, I think Lewis's analysis is much stronger than Faux's. I'm happy I read both books.
Rating: 8 out of 10
parallel @ Savannah: GNU Parallel 20241222 ('Bashar') released [stable]
GNU Parallel 20241222 ('Bashar') has been released. It is available for download at: lbry://@GnuParallel:4
Quote of the month:
"Do this with gnu parallel" is the Copilot hack of the day
-- Chase Clark @chasingmicrobes.bsky.social
New in this release:
- No new features. This is a candidate for a stable release.
- Bug fixes and man page updates.
GNU Parallel - For people who live life in the parallel lane.
If you like GNU Parallel record a video testimonial: Say who you are, what you use GNU Parallel for, how it helps you, and what you like most about it. Include a command that uses GNU Parallel if you feel like it.
GNU Parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU Parallel can then split the input and pipe it into commands in parallel.
If you use xargs and tee today you will find GNU Parallel very easy to use as GNU Parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU Parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. GNU Parallel can even replace nested loops.
GNU Parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU Parallel as input for other programs.
For example you can run this to convert all jpeg files into png and gif files and have a progress bar:
parallel --bar convert {1} {1.}.{2} ::: *.jpg ::: png gif
Or you can generate big, medium, and small thumbnails of all jpeg files in sub dirs:
find . -name '*.jpg' |
parallel convert -geometry {2} {1} {1//}/thumb{2}_{1/} :::: - ::: 50 100 200
You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/
You can install GNU Parallel in just 10 seconds with:
$ (wget -O - pi.dk/3 || lynx -source pi.dk/3 || curl pi.dk/3/ || \
fetch -o - http://pi.dk/3 ) > install.sh
$ sha1sum install.sh | grep 883c667e01eed62f975ad28b6d50e22a
12345678 883c667e 01eed62f 975ad28b 6d50e22a
$ md5sum install.sh | grep cc21b4c943fd03e93ae1ae49e28573c0
cc21b4c9 43fd03e9 3ae1ae49 e28573c0
$ sha512sum install.sh | grep ec113b49a54e705f86d51e784ebced224fdff3f52
79945d9d 250b42a4 2067bb00 99da012e c113b49a 54e705f8 6d51e784 ebced224
fdff3f52 ca588d64 e75f6033 61bd543f d631f592 2f87ceb2 ab034149 6df84a35
$ bash install.sh
Watch the intro video on http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Walk through the tutorial (man parallel_tutorial). Your command line will love you for it.
When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2018): GNU Parallel 2018, March 2018, https://doi.org/10.5281/zenodo.1146014.
If you like GNU Parallel:
- Give a demo at your local user group/team/colleagues
- Post the intro videos on Reddit/Diaspora*/forums/blogs/ Identi.ca/Google+/Twitter/Facebook/Linkedin/mailing lists
- Get the merchandise https://gnuparallel.threadless.com/designs/gnu-parallel
- Request or write a review for your favourite blog or magazine
- Request or build a package for your favourite distribution (if it is not already there)
- Invite me for your next conference
If you use programs that use GNU Parallel for research:
- Please cite GNU Parallel in you publications (use --citation)
If GNU Parallel saves you money:
- (Have your company) donate to FSF https://my.fsf.org/donate/
GNU sql aims to give a simple, unified interface for accessing databases through all the different databases' command line clients. So far the focus has been on giving a common way to specify login information (protocol, username, password, hostname, and port number), size (database and table size), and running queries.
The database is addressed using a DBURL. If commands are left out you will get that database's interactive shell.
When using GNU SQL for a publication please cite:
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
GNU niceload slows down a program when the computer load average (or other system activity) is above a certain limit. When the limit is reached the program will be suspended for some time. If the limit is a soft limit the program will be allowed to run for short amounts of time before being suspended again. If the limit is a hard limit the program will only be allowed to run when the system is below the limit.
Oliver Davies' daily list: How easily can you move changes between environments?
Regardless of how many environments your application has, you need to be able to move changes between them reliably.
You don't want to configure each environment and make every change by hand.
You want to automate this as much as possible so your changes are the same every time.
In Drupal 7, the Features module was used to export changes once and apply them again using a features revert command - although its original use case was to extract reusable features for different applications.
I've also written a lot of update of update hooks, like mymodule_update_8001 to apply changes when database updates are applied.
Since Drupal 8, we've had configuration management - a first-class way to export and import configuration changes - which I think was one of the best additions to Drupal 8, and something not available in some other CMSes, frameworks and applications.
There's an ecosystem around configuration management, including Config Split for per-environment configurations and Config Ignore to ignore sensitive information or changes you don't want to manage via imported configuration.
I recently worked on a project where we didn't have a CI pipeline running configuration imports on each change and things were very difficult to manage. Once that was in place, though, things were much easier, more consistent and changes were quicker to release.
gtypist @ Savannah: GNU Typist 2.10 released
This is a major release
Changes in 2.10:
- new welcome screen
- new P lesson series for programmers
- fixes for various lessons
- new Romanian lessons
- expand the S lesson series with a new quotation and a few more passages from Shakespeare
- jump over whitespace characters at the beginning of lines in lessons
- fix the terminal resize bug
- fix a few compilation warnings
- add the Romanian translation
- updates to a few translations
- few updates to the documentation
- update the project license to GPL3+
- remove or update the lessons incompatible with the new license
- update the KTouch lesson import script
- fix warnings from help2man generated manual pages
- fix a few comments
Sources for this release can be downloaded here:
https://ftp.gnu.org/gnu/gtypist/gtypist-2.10.tar.gz
Top Open Source licenses in 2024
The Open Source Initiative (OSI) serves as the premier resource for millions of visitors seeking essential information about OSI-Approved licenses. The enriched license pages go beyond basic descriptions, incorporating relevant metadata to provide deeper insights and better support for Open Source users, developers, and organizations.
The most popular licenses include the MIT license, BSD licenses (3-clause and 2-clause), Apache 2.0 license, and GNU General Public license (2.0 and 3.0). These licenses continue to lead the way as the go-to choices for countless Open Source projects worldwide, reflecting their widespread adoption and versatility.
Here’s the top 20 OSI-Approved licenses most frequently sought out by our community in 2024 based on number of pageviews.
LicensePageviewsVisitorsmit1.7M1Mbsd-3-clause247.9K207Kapache-2-0244.6K184Kbsd-2-clause115.8K98.2Kgpl-2-083.8K71.7Kgpl-3-072.4K61.5Kisc-license-txt48.7K34.9Klgpl-2-121.6K19Klgpl-3-017.8K15.4KOFL-1.116.9K14.1Kmpl-2-014.8K12.8Kpostgresql14.6K12Kms-pl-html13.8K9.9Kagpl-v313.3K11.1K0bsd13.1K11.1Kbsd-1-clause10.3K8.7Kafl-3-0-php10.1K8.6Kcpl1-0-txt8.1K5.9Kosl-3-0-php7.2K6.3Kmit-06.7K5.8KIn 2025, we plan to further enhance these license pages with even more features, insights, and user-focused improvements. Whether you’re a seasoned developer, a project maintainer, or just beginning your Open Source journey, these resources will continue to evolve to meet your needs. If you are interested in supporting our work, please consider donating to or sponsoring the OSI.
Russ Allbery: Dropping the git protocol
Ever since I started converting my free software repositories to Git and hosting them on git.eyrie.org, I've made them available via the Git network protocol (the one that uses git:// URLs). This protocol doesn't support TLS or any other form of integrity checking on the wire and has been gently deprecated for some time. Recent changes to git daemon broke the way I was managing permissions for my personal repositories, which meant that the git://git.eyrie.org/ URLs have been broken for several months. Since no one has complained or apparently noticed, this seems like a good excuse to drop support entirely and run one fewer service.
All of my personal repositories will continue to be accessible via Git over HTTPS, which is now the standard for anonymous Git checkouts. If the prior URL was git://git.eyrie.org/area/package.git, the new URL is https://git.eyrie.org/git/area/package.git. I've updated my web pages accordingly. This protocol provides integrity protection on the wire and a moderate amount of authentication of my server via a Let's Encrypt certificate.
Daniel Roy Greenfeld: TIL: SequentialTaskSet for Locust
SequentialTaskSet makes it so Locust tasks happen in a particular order, which ensures your simulated users are clicking around in a more human manner at a more human pace. Attribution goes to Audrey Roy Greenfeld.
You can see it in action in the now updated previous entry on the topic of Locust for load testing.
Daniel Roy Greenfeld: TIL: Making pytest use Ipython's PDB
Usage:
pdb tests/test_things::test_broken_thingPages
Recent Publications
- Managing Hidden Dependencies in OO Software: a Study Based on Open Source Projects
- Open Source Communities as Liminal Ecosystems
- Investigating developers' email discussions during decision-making in Python language evolution
- Developers, Quality Control and Download Volume in Open Source Software (OSS) Projects
More...