FLOSS Project Planets

Bryan Pendleton: It looked like nothing, and that was on purpose

Planet Apache - Sat, 2015-11-14 14:39

A nifty short article on Wired about the demolition of E3: Watch Part of the Old Bay Bridge Implode.

Just after a lovely sunrise, a series of muffled booms shot across the San Francisco Bay, and a plume of water swallowed a huge chunk of the old Bay Bridge.

The chunk was once the E3 pier, a 20 million-ton concrete strut that reached from the water’s surface to the a foundation 50 feet below. At 7:18 am this morning 60,000 pounds of dynamite crumbled it into a hollow cylinder encased in Bay mud.

As the article goes on to explain, this was (deliberately) a very tame demolition from a spectator point of view. I was quite interested in this part:

Those two barges were a key part of the joy-killing efforts to conserve local wildlife. A series of hoses deployed from each sprayed the underwater portion of the pier in bubbles. "The bubble curtain is to contain the shock wave from the implosion," said Leah Robinson-Leach, CalTrans’ spokesperson for all things San Francisco Bay Bridge. To further spoil the fun (or protect people and structures safe from flying debris, again, depending on your perspective), the 80-by-140 foot rectangular top of the pier was covered by a huge steel and wood mat.

"Bubble curtain"? COOL!

I wonder if there is any underwater video of that part?

It is super-important to keep the San Francisco Bay as clean as possible, which is a real challenge given all the uses of the Bay.

So I'm pleased to see Cal Trans do what they can to avoid making matters worse, whenever possible.

And it was a cool little video in the article -- check it out!

Categories: FLOSS Project Planets

Justin Mason: Links for 2015-11-13

Planet Apache - Fri, 2015-11-13 18:58
  • The impact of Docker containers on the performance of genomic pipelines [PeerJ]

    In this paper, we have assessed the impact of Docker containers technology on the performance of genomic pipelines, showing that container “virtualization” has a negligible overhead on pipeline performance when it is composed of medium/long running tasks, which is the most common scenario in computational genomic pipelines. Interestingly for these tasks the observed standard deviation is smaller when running with Docker. This suggests that the execution with containers is more “homogeneous,” presumably due to the isolation provided by the container environment. The performance degradation is more significant for pipelines where most of the tasks have a fine or very fine granularity (a few seconds or milliseconds). In this case, the container instantiation time, though small, cannot be ignored and produces a perceptible loss of performance.

    (tags: performance docker ops genomics papers)

Categories: FLOSS Project Planets

Gary Gregory: All about Java 9

Planet Apache - Fri, 2015-11-13 18:19

These videos are all about Java 9 from the Devoxx conference:

Thanks to Arnaud Héritier for gathering these links for the Maven dev mailing list.

Categories: FLOSS Project Planets

Claus Ibsen: Using Camel commands to manage a Spring Boot Camel application

Planet Apache - Fri, 2015-11-13 12:48
So its friday and I though I could do a quick video recoding of a new feature coming in Apache Camel 2.17.

Thanks to community contribution we have the Camel commands ported to work with Spring Boot. That means you can just add camel-spring-boot and hack your Camel routes and whatnot, and included you get the Camel commands.

To use the Camel commands you need to enable the Spring shell, which is just to add a dependency


Then when you run spring-boot then it has shell enabled out of the box. You can then SSH into the JVM of the spring-boot and use its shell to manage it. And in there you now have the camel commands as well.

The video shows all this in action.

And you can try yourself with the camel-example-spring-boot which we ship in the Apache Camel release. All this will be in the Camel 2.17 release which is expected in Q1 2016 (may slip into start of Q2).

The link to the video is here.
Categories: FLOSS Project Planets

Joe Brockmeier: Polar madness – bears everywhere!

Planet Apache - Fri, 2015-11-13 07:24
Still in vacation mode, but wanted to share a few quick photos from the trip. Enjoy!
Categories: FLOSS Project Planets

Bryan Pendleton: In which people discuss things I don't understand

Planet Apache - Thu, 2015-11-12 22:36

The first link is REALLY worth a read ... very eye-opening and intriguing.

  • Uber's Drivers: Information Asymmetries and Control in Dynamic WorkThe Uber driver workplace is characterized by constant change and by remote management structures, such as algorithms, Community Support Representatives, and passengers, removes the governing responsibility for a reliable workplace away from a central actor – Uber as a corporate entity, or a singular managerial body. Drivers must compare the information they gather from their own experiences with CSRs, media reports, company statements, written policies, notices from local markets, and their own advice in forums as though there is a singular, sense-making machine at work. There are multiple authorities for what Uber says or does that drivers rely on because the Uber system provides the architecture for digital and physical points of engagement and interaction with different authoritative actors. As a case study in the emerging on-demand economy, our analysis of the Uber driver experience signals the need for further study of the social and technical dynamics of distributed work systems.
  • The Guilded AgeUber’s regulatory battles will, to some extent, pave the way for other services, be they car-hailing apps or delivery networks or privatized replacements for public transit or just other types of on-demand labor whatevers. Airbnb’s will free up, to some extent, Airbnb competitors. But because they’re first, and because they’re huge, and because their investors have lots of adjacent interests, these regulatory battles belong to them. This means our next laws regarding how people drive and get driven, and the next sets of rules determining what and where a hotel can be, will be written largely by these companies.
  • Airbnb Is Building An ArmyAirbnb policy staffers are already on the ground around the world. Lehane said in cities where clubs are founded, staffers will be called on to organize training, facilitate resources, and otherwise manage the beginnings of an international grassroots network. While most cities don’t have the same proposition system as San Francisco, which allows voters to weigh in directly on initiatives, he said he could foresee clubs supporting political candidates who are in favor of short-term renting and home-sharing in their cities. Lehane compared the potential political strength of Airbnb hosts and guests, of which there are over 4 million in the United States, to that of the National Rifle Association or the Sierra Club.
  • Peeking Beneath the Hood of UberIn order to understand the impact of surge pricing on passengers and drivers, we present the first in-depth investigation of Uber. We gather four weeks of data from Uber by emulating 43 copies of the Uber smartphone app and distributing them in a grid throughout downtown San Francisco (SF) and midtown Manhattan. By carefully calibrating the GPS coordinates reported by each emulated app, we are able to collect high-fidelity data about surge multipliers, estimated wait times (EWTs), car supply, and passenger demand for all types of Ubers (e.g., UberX, UberBLACK, etc.).
Categories: FLOSS Project Planets

Justin Mason: Links for 2015-11-12

Planet Apache - Thu, 2015-11-12 18:58
Categories: FLOSS Project Planets

Ian Boston: What do do when your ISP blocks VPN IKE packets on port 500

Planet Apache - Thu, 2015-11-12 10:16

VPN IKE packets are the first phase of establishing a VPN. UDP versions of this packet go out on port 500. Some ISPs (PlusNet) block packets to routers on port 500, probably because they don’t want you to run a VPN end point on your home router. However this also breaks a normal 500<->500 UDP IKE conversation.  Some routers rewrite the source port of the IKE packet so that they can support more than one VPN. The feature is often called a IPSec application gateway. The router keeps a list of the UDP port mappings using the MAC address of the internal machine. So the first machine to send a  VPN IKE packet will get 500<->500, the second 1500<->500, the third 2500<->500 etc. If your ISP filters packets inbound to your router on UDP 500 the VPN on the first machines will always fail to work.  You can trick your router into thinking your machine is the second or later machine by changing the MAC address before you send the first packet. On OSX

To see the current MAC address use ifconfig, and take a note of it.

then on the interface you are using to connect to your network do

sudo ifconfig en1 ether 00:23:22:23:87:75

Then try and establish a VPN. This will fail, as your ISP will block the response to your port 500. Then reset your MAC address to its original

sudo ifconfig en1 ether 00:23:22:23:87:74

Now when you try and establish a VPN it will send a IKE packet out on 500<->500. The router will rewrite that to 1500<->500 and the VPN server will respond 500<->1500 which will get rewritten to 500<->500 with your machine IP address.

How to debug

If you still have problems establishing a VPN then using tcpdump will show you what is happening. You need to run tcpdump on the local machine and ideally on a network tap between the router and the modem. If you’re on Fibre or Cable, then a Hub can be used to establish a tap. If on ADSL, you will need something harder.

On your machine.

sudo tcpdump -i en1 port 500

On the network tap, assuming eth0 is unconfigured and tapping into the hub. This assumes that your connection to the ISP is using PPPoE. Tcp will decode PPPoE session packets, if you tell it to.

sudo tcpdump -i eth0 -n pppoes and port 500

If your router won’t support more than 1 IPSec session, and uses port 500 externally, then you won’t be able to use UDP 500 IKE unless you can persuade your ISP to change their filtering config.

Categories: FLOSS Project Planets

Edward J. Yoon: Tensorflow 알아보자

Planet Apache - Thu, 2015-11-12 06:38

글보다는 우선 그림만 먼저보는 나이기에 가장 먼저 눈에 띈 놈 Figure 2. 첨부.
W는 weight matrix, X는 training instance 겠고, b는 bias겠지. MatMul하고 b 추가하고 ReLu 활성 함수  f(x) = Max(0, x) 통과시키고 있는 장면이리라. C는 콘볼루쇼널에 C겠지.

여기서 느끼겠지만 이놈의 장점은 편한 Graph 스타일이라는거다. 각 노드를 체이닝해서 처리하는데 (그래서 Flow) 이때 각 노드는 특정 디바이스 활용한다던가? 그렇고 노드간 통신도 가능 (통신 오버헤드 고려 옵티마이제이션은 어카지?).

여기서 또 한가지 의문은 모델 페러럴을 하려면 이런식으로 레이어 기준 병렬화를 사용자가 정의해야되는데 그럼 너무 복잡한거 아닌가 싶다. 모델 구조에 따라 파티셔닝 전략도 달라져야할 필요가 있는데 그런 경우는 또 어칼꺼?
Categories: FLOSS Project Planets

Bryan Pendleton: Quite a bit more than the whole 9 yards...

Planet Apache - Wed, 2015-11-11 21:28

In the annals of construction porn, this is an oddly mesmerizing little gem: an 8.5 minute condensed time-lapse video of the 16 hour foundation pour of the new Salesforce.com super-skyscraper that's now underway in the heart of San Francisco.

Wired reports with more details: It Took 18 Hours to Pour San Francisco's Biggest-Ever Concrete Foundation

All that concrete did not slop down into an earthy void. In the weeks leading up to the pour, workers constructed a subterranean lattice of rebar—12 layers high, with six inches separating each layer. "We used 5 million pounds of number 18 rebar, the largest size available," says Tymoff. At two and a quarter inches in diameter, grabbing a bar of No. 18 is like gripping the fat end of a baseball bat. It took eight iron workers to lift each 45-foot segment into place. That cagework will act as the foundation’s skeleton, but during the pour it also served as a catwalk for workers holding the cement hoses or the massive vibrators used to ensure the concrete had no air pockets.

The hardened slab, in all its mightiness, is but half of the tower’s earthquake protection. It will keep the building from toppling sideways, but what about sliding back and forth? In a big earthquake, the ground is actually trying to slip sideways underneath the building. "You need something to keep you from changing addresses," says Joseph. Those somethings are called piles, in essence underground stilts connecting the building with the bedrock. In the lowlands of San Francisco’s Financial District, bedrock is 300 below street level. "We have 42 piles that go all the way down and are socketed 15 to 25 foot deep into the rock," says Tymoff.

And of course, for even MORE detail, don't miss the wonderful site run by architects Pelli Clarke Pelli: Salesforce Tower.

At its base, Salesforce Tower connects directly to the transit center, which will house 11 Bay Area transit systems. On top of the Transit Center and linked directly to the tower is a 5.4-acre public park, which will offer recreational, educational, and nature activities. The park has two roles: the future anchor of the neighborhood and a key element of the project’s sustainable design strategy.

Each floor of the tower will have integrated metal sunshades, calibrated to maximize light and views while reducing solar gain. High performance, low-​emissivity glass will also help to reduce the building’s cooling load. Cooling may be provided in part by heat-​exchanging coils wrapped around the tower’s foundations. The tower and transit center also include comprehensive water recycling systems. In addition, high efficiency air-​handlers will take in fresh air on every floor.

Or, if you just can't stand it, head on over to the skyscraper's own website run by Boston Properties, and keep up with the minute-by-minute progress on Construction Cam!

It's interesting how these giant construction projects go. A few years back, I was completely fascinated by the new Bay Bridge, and in particular by the custom barge-based floating crane that was commissioned and delivered especially for the project: the Left Coast Lifter.

Now the bridge is built (and in fact the old bridge is pretty well completely torn down and removed), and the Left Coast Lifter hasn't been around these parts for years.

But last week, when I was in New York, we happened to make a side trip (to Storm King -- that reminds me, I need to blog about that, too!), and we found ourselves crossing the Tappan Zee Bridge.

And there, what did I see to my delighted eyes?

It's the Left Coast Lifter!

Alive and well, it's happily sitting in the Hudson River in upstate New York, contentedly building the new bridge.

The folks on that side of the country call it the I Lift NY supercrane.

But as we whizzed by on the super-speedway I could still make out the words painted across the bow:

Left Coast Lifter

So there you go.

Categories: FLOSS Project Planets

Justin Mason: Links for 2015-11-11

Planet Apache - Wed, 2015-11-11 18:58
  • Dynalite

    Awesome new mock DynamoDB implementation:

    An implementation of Amazon’s DynamoDB, focussed on correctness and performance, and built on LevelDB (well, @rvagg’s awesome LevelUP to be precise). This project aims to match the live DynamoDB instances as closely as possible (and is tested against them in various regions), including all limits and error messages. Why not Amazon’s DynamoDB Local? Because it’s too buggy! And it differs too much from the live instances in a number of key areas. We use DynamoDBLocal in our tests — the availability of that tool is one of the key reasons we have adopted Dynamo so heavily, since we can safely test our code properly with it. This looks even better.

    (tags: dynamodb testing unit-tests integration-testing tests ops dynalite aws leveldb)

  • Alarm design: From nuclear power to WebOps

    Imagine you are an operator in a nuclear power control room. An accident has started to unfold. During the first few minutes, more than 100 alarms go off, and there is no system for suppressing the unimportant signals so that you can concentrate on the significant alarms. Information is not presented clearly; for example, although the pressure and temperature within the reactor coolant system are shown, there is no direct indication that the combination of pressure and temperature mean that the cooling water is turning into steam. There are over 50 alarms lit in the control room, and the computer printer registering alarms is running more than 2 hours behind the events. This was the basic scenario facing the control room operators during the Three Mile Island (TMI) partial nuclear meltdown in 1979. The Report of the President’s Commission stated that, “Overall, little attention had been paid to the interaction between human beings and machines under the rapidly changing and confusing circumstances of an accident” (p. 11). The TMI control room operator on the day, Craig Faust, recalled for the Commission his reaction to the incessant alarms: “I would have liked to have thrown away the alarm panel. It wasn’t giving us any useful information”. It was the first major illustration of the alarm problem, and the accident triggered a flurry of human factors/ergonomics (HF/E) activity. A familiar topic for this ex-member of the Amazon network monitoring team…

    (tags: ergonomics human-factors ui ux alarms alerts alerting three-mile-island nuclear-power safety outages ops)

  • An Analysis of Reshipping Mule Scams

    We observed that the vast majority of the re-shipped packages end up in the Moscow, Russia area, and that the goods purchased with stolen credit cards span multiple categories, from expensive electronics such as Apple products, to designer clothes, to DSLR cameras and even weapon accessories. Given the amount of goods shipped by the reshipping mule sites that we analysed, the annual revenue generated from such operations can span between 1.8 and 7.3 million US dollars. The overall losses are much higher though: the online merchant loses an expensive item from its inventory and typically has to refund the owner of the stolen credit card. In addition, the rogue goods typically travel labeled as “second hand goods” and therefore custom taxes are also evaded. Once the items purchased with stolen credit cards reach their destination they will be sold on the black market by cybercriminals. [...] When applying for the job, people are usually required to send the operator copies of their ID cards and passport. After they are hired, mules are promised to be paid at the end of their first month of employment. However, from our data it is clear that mules are usually never paid. After their first month expires, they are never contacted back by the operator, who just moves on and hires new mules. In other words, the mules become victims of this scam themselves, by never seeing a penny. Moreover, because they sent copies of their documents to the criminals, mules can potentially become victims of identity theft.

    (tags: crime law cybercrime mules shipping-scams identity-theft russia moscow scams papers)

Categories: FLOSS Project Planets

Bryan Pendleton: Phil Lesh and Friends, Capitol Theatre, Port Chester, NY, Nov 6 &amp; 7, 2015

Planet Apache - Wed, 2015-11-11 11:44

Nowadays, in the world of music entertainment, the typical band has a stable and well-known roster of performers; those performers are the primary reason you choose to attend one performance versus another, after all.

And the typical tour has a stable and predictable program, drawn from an obvious repertoire; the most common justification for a tour nowadays is to promote a new album release, and hence it's almost universal to expect that the performance will consist primarily of material from that new release.

Phil Lesh is typical in neither way.

Lesh, one of the founders and the former bass player for the Grateful Dead, has evolved a most interesting and unusual performance style which he calls "Phil Lesh and Friends".

After more than 50 years as a professional musician, Lesh has an enormous number of musical contacts, as well as an extensive and diverse collection of material.

There are even those who give him credit for how the band got its name:

What matters for our purposes is that Wenner, arguably the 20th century’s most important and influential rock journalist and publisher, got his scoop on the band’s name directly from the its bassist, Phil Lesh, who played an important role in giving the band its name—it was at Lesh’s home that Jerry Garcia came upon the phrase "The Grateful Dead" in "a big Oxford Dictionary," as Garcia remembers it in Signpost. That may be why the name was so fresh in Lesh’s mind when he told Wenner "We’re the Grateful Dead."”

And, perhaps most importantly, he has wide-ranging interests and a genuine joy of performance, which drive him to find ways to continue playing and interacting with his audience.

So, his format (roughly) is this: every so often, Lesh contacts some number of his friends, who clear time on their schedule, and make some arrangements to meet and discuss and prepare.

Then, at the appointed date, and at the appointed location, Phil Lesh and Friends appear, and deliver their show.

It's never the same show twice.

You never know ahead of time who's going to be in the band (except, of course, for Lesh).

And you never know ahead of time what will be on the program (although, broadly speaking, you know what sort of material it will be, since after 50 years everybody knows what sort of music Lesh enjoys).

It's a pretty unusual format. And, given that Lesh is now 75 years old, and has had numerous health problems (liver transplant, prostate cancer, bladder cancer), you never know how much longer you might get a chance to see him in action.

And so it came to be that, mostly as an excuse for a far-too-long-postponed visit to my very oldest and dearest friends on the planet, we hopped on the plane and I came to be in Port Chester, New York, on November 6th and 7th, 2015.

It's worth, as a side-note, mentioning why, specifically, we were in Port Chester. Although the Capitol Theatre was famous, 45 years ago, as the site of some of The Grateful Dead's most famous shows, the theater had become disused and was idle until recently. However, as part of its re-opening, Lesh was named "musician in residence" and has been playing there regularly, and it is clearly one of his favorite places to play.

It's hard to over-state the difference between seeing a show at the Capitol Theatre versus almost any other that shows acts of this caliber. The theater holds fewer than 2,000 people; in contrast, when I saw The Grateful Dead in June here in California, there were nearly 50 times as many people in the audience, and we spent the time watching the show on 70-foot-tall video screens, for the most part.

But at the Capitol, the space is small and friendly and personal. You can almost imagine that you have been invited into their living room and you are sitting on the couch, listening to them play and sing and talk and relax.

Well, you and your 1,799 new best friends, that is.

So, on to the music.

During this particular event, Phil Lesh's friends were David Nelson, Barry Sless, Scott Law, Jason Crosby, and John Molo.

Of those musicians, David Nelson is probably the most famous. He might be best known for his group New Riders of the Purple Sage, but he also played with the Grateful Dead many a time back in the day; for instance, he plays the electric guitar on the recording of Jack Straw on American Beauty. As the group's web site recalls:

In the summer of 1969, John Dawson was looking to showcase his songs while Jerry Garcia was looking to practic his brand new pedal steel guitar. The two played in coffeehouses and small clubs initially, and the music they made became the nucleus for a band - the New Riders of the Purple Sage.

That same year, David Nelson, expert in both country and rock guitar, joined the group on electric lead guitar.

As you might expect from this line-up, the selection of music for the two shows included several New Riders songs: John Hardy's Wedding, Garden of Eden, Bob Dylan's The Wicked Messenger, and of course their signature song, The Adventures of Panama Red.

And naturally there were a broad range of Grateful Dead signature songs, including several of Lesh's own compositions: Box of Rain (which is a personal favorite of mine), Pride of Cucamonga, Mason's Children (which disappeared from the Dead's regular rotation long before I started following them closely), and Unbroken Chain, as well as Grateful Dead classics not so closely linked with Lesh, such as Jack Straw (led, as a delightful surprise, by Lesh's son Grahame Lesh on vocals and guitar), Uncle John's Band, Dire Wolf, Cold Rain and Snow, and Scarlet Begonias

Perhaps because of the musicians that were particularly present for these concerts, the music selection also drew from the Grateful Dead's long history of blues, bluegrass, boogie-woogie, and jug band traditions, including pieces such as Goin' Down the Road Feeling Bad, White Lightning, Loose Lucy, Turn On Your Love Light, and Not Fade Away

But most interesting of all, musically, I think, was the inclusion of three fairly unusual songs from an American musician who is not so well known at all, I think: Noah Lewis. Let's let AllMusic.com's biography of Lewis tell some of his story:

A key figure on the Memphis jug band circuit of the 1920s, singer and harpist Noah Lewis was born on September 3 of either 1890 or 1895 (depending on sources) in Henning, Tennessee. Upon relocating to Memphis, he teamed with Gus Cannon, becoming an essential component of Cannon's Jug Stompers; the group made their debut recordings for the Paramount label in 1927, with several more sessions to follow prior to their final date in late 1930. On a series of sides cut in the first week of October 1929, Lewis made his debut as a name artist, cutting three blistering harmonica solos as well as "Going to Germany," which spotlighted his plaintive vocal style. Later recording with Yank Rachell and John Estes, as the Depression wore on Lewis slipped into obscurity, living a life of extreme poverty; his death on February 7, 1961 was a result of gangrene brought on by frostbite.

Lewis died before I was even born; this is OLD music as far as things go in American music history.

What is the "Jug Band"? Well, again, let's turn to AllMusic.com's site for more information:

Jug bands united Appalachian folk with blues, ragtime, and very early jazz; they are best known, of course, for their novel, do-it-yourself instrumentation. The jug in question was usually a whiskey jug, and a player blew across the mouth of the jug to produce pitches in the bass register. Jug bands usually featured at least one stringed instrument from the Appalachian tradition -- guitar, banjo, and/or fiddle -- and used a wide variety of everyday, easily available household objects for rhythmic accompaniment. The most common were the washboard (whose slats were struck and rubbed in a way analogous to a snare drum) and the metal washtub bass, which was usually equipped with a broomstick and clothesline that produced the sounds. Other possible percussion instruments included spoons, gut buckets, bones, and saw blades; additional melodic accompaniment might have included a harmonica, kazoo, or even comb and tissue paper -- whatever was available and economical, really. Jug band music originated in Louisville, Kentucky at the dawn of the 1900s, but found its greatest popularity in Memphis, Tennessee during the '10s and '20s, eventually spreading to Ohio and North Carolina as well. Given the inherent playfulness of the instrumentation, jug band music was accordingly informal, spontaneous, often humorous, and rhythmically bouncy.

Jug Band music and The Grateful Dead have gone together for at least 50 years, of course, but it was quite pleasing to me to see the selection of three of Noah Lewis's pieces in Saturday's program: Minglewood Blues, Big Railroad Blues, and Viola Lee Blues.

The three songs have some interesting differences. Big Railroad Blues is a crowd-pleasing sing-along, a blues song with a light-hearted sense of irony and and humor, as our hero sings:

Well my mama told me, my papa told me too,
Now my mama told me, papa told me too,
Well I shouldn't be here tryin' to sing these railroad blues.

Wish I had a'listened to what my mama said,
Wish I had a'listened to what my mama said,
Well I wouldn't be here tryin' to sleep in this cold iron bed.

Minglewood Blues, on the other hand, as my co-worker "reb" notes, has a long and complex history. What or where is "Minglewood"? Well, The Old Weird America takes a stab at clarifying it:

The answer to where exactly is this "Minglewood" is a bit uncertain. I have read somewhere that it was a lumber camp near the Mississippi where musicians (including Noah lewis and Gus cannon) gathered on weekends to have a good time, and judging from the lyrics of the "New Minglewood Blues" that Noah Lewis recorded with his own jug band ("If you’re ever in Memphis, better stop by Minglewood"), it was a place in the city or close to it. Make sure you follow the link from that site to The Myth of Minglewood for more backstory about Minglewood.

But, for my tastes, the best of the three Noah Lewis songs was the third one, the heart-breaking and beautiful Viola Lee Blues. This song meant so much to Lesh and the rest of The Grateful Dead that they made it the 10 minute long epic climax of their first actual album.

The wonderful Grateful Dead Guide site discusses the 50 year history of The Grateful Dead's history with the song, and how it morphed and evolved through the years; it's clear that it meant a tremendous amount to them.

Viola Lee Blues was the Dead's first big jamming tune. Dating from the start of their career when they were doing mostly pop and blues songs, they designed it as a psychedelic trip: it would start as a strange old jugband tune with dark chords, a constricted groove, and wailing black-harmony vocals, but the music in-between the verses would gradually stretch out to unreasonable lengths and start accelerating until the band were playing fast, shrieking gusts of sound, tearing open the fabric of reality -- then suddenly the noise stops and the song jauntily reappears again. As one writer has said, it may have been a one-dimensional song, but that happened to be the fifth dimension!

You can find many of The Grateful Dead renditions on the net, of course, but here's an alternate suggestion instead: listen to this wonderful performance by Jim Kweskin's Jug Band while you follow on with the lyrics here, since they're a bit hard to make out until you've heard the song a few hundred times:

The judge decreed it, the clerk he wrote it.
Clerk he wrote it down indeed-e
Judge decreed it, clerk he wrote it down
Give you this jail sentence you'll be Nashville bound

Some got six month some got one solid.
Some got one solid year indeed-e
Some got six month some got one solid.
But me and my buddies all got lifetime here

I wrote a letter I mailed in the air,
Mailed it on the air indeed-e
I wrote a letter I mailed in the air.
You may know by that I've got a friend somewhere

It's nearly a hundred years since Noah Lewis penned his sorrowful, tragic, heart-breaking tale of injustice, loneliness, and despair, but that amazing, glorious final verse, with its simple recognition that that simple act of writing a letter, of reaching out, of trying to communicate with some other human being somewhere else, has the power to overcome that cruelty and show the world that "I've got a friend somewhere."

This is

fast, shrieking gusts of sound, tearing open the fabric of reality indeed.

So even though there were lots of wonderful, wonderful things to remember about these shows, the sentiment of this majestic hundred-year-old song appealed to me on my madcap visit to my friends-of-four-decades, and somehow it seemed the perfect way for me to try to make sense of the entire experience.

Categories: FLOSS Project Planets

Colm O hEigeartaigh: Testing Kerberos with Web Services using Apache Kerby

Planet Apache - Wed, 2015-11-11 07:13
The previous blog post described how to use Apache Directory to easily create a KDC via Java annotations for kerberos integration testing. In this post, we will look at an alternative way of setting up a KDC for integration testing using Apache Kerby.

Apache Kerby is a new subproject of Apache Directory that aims to provide a complete Kerberos solution in Java. Version 1.0.0-RC1 has recently been released and is available for testing. Apache Kerby consists of both a KDC as well as a client API, that is completely independent of the GSS API that comes with Java. A key selling point of Apache Kerby is that it is very easy and fast to setup and deploy a KDC. It is possible to set up a KDC completely in code, without having to edit any configuration files or configure any system properties.

Let's see how this is done by looking at a project I created on github:
  • cxf-kerberos-kerby: This project contains a number of tests that show how to use Kerberos with Apache CXF, where the KDC used in the tests is based on Apache Kerby.
The KDC is launched in the test-code, and is pretty much as straightfoward as the following code snippet:

The first block of code configures the host, realm, transports and ports, while the second creates the client, service and TGT principals that are used in the tests. No configuration files required! As well as showing how to use Apache CXF to authenticate using both Kerberos and Spnego for a JAX-WS service, the AuthenticationTest also includes unit tests for getting a service ticket from the Kerby KDC using the Java GSS API as well as the Kerby client API. Using the Kerby client API is as simple as this:

Have fun playing around with Apache Kerby and please join and contribute to the project if you are interested!
Categories: FLOSS Project Planets

Claus Ibsen: I am presenting Apache Camel in Copenhagen next week

Planet Apache - Wed, 2015-11-11 04:03
Next week I am going home to Denmark, to present Apache Camel in general and then a 2nd part that is focused around developing micro services with Camel, and running those in Docker containers in a Kubernetes platform. And all of that becomes easier for developers with the help from the tools and services we provide in the fabric8 project.

The event takes place on tuesday 17th november in the afternoon. There is three sessions, where Jakob Bendsen and Christian Damsgaard will talk about APIs and RESTful services with Camel.

The agenda, location, and how to register is all provided by Javagruppen, whom is organizing the event.

You can find the details here.

After all the talks there is pizza and beverages. Hope you have the time to stay, as I loved to hear war stories from the fields, and potentially news about where Camel's are in use. I have also some great stories about where Camel are in use you may not know or realize how a prominent role it plays in so many companies and public sections around the globe

The location was initially in Glostrup, but the event was quickly sold out (its free to attend for Javagruppen members) in less than 12 hours. So Region Hovedstaden was quick to save the day and provide a bigger location.

PS: The event takes place at Region Hovedstaden, Center for It, Medico og Telefoni, where I in the past had worked as a consultant. It's great to be back for a day. Hope to see familiar faces at the event. Region Hovedstaden is a long time Apache Camel user.

Categories: FLOSS Project Planets

Bryan Pendleton: New York City, Fall 2015

Planet Apache - Tue, 2015-11-10 21:36

It came to pass that we had the opportunity to spend 48 hours in Manhattan, wandering around and enjoying ourselves.

And so we did.

Manhattan is so big and complex that it would take months, years, perhaps your entire life, in order to really understand it.

But we didn't have that; we just had 48 hours.

So we had to concentrate, and pick a few things.

It so happened that we arrived in New York fairly late in the afternoon. By the time we had checked in to our hotel, it was already dinner time, so we went down to a nice (although quite busy) little restaurant in the East Village (just off St Mark's Place) for a nice meal.

After dinner we got back to the hotel, but we weren't quite ready to call it quits, so we went up to the 48th floor, where the revolving rooftop bar made a delightful location to have a drink before bed. It's a great experience; a fun touch is that the cocktail napkins are printed with a circular "skyline map" identifying all the buildings that you see, so that as you rotate around you can make sense of what you're looking at.

Assuming you're brave enough to actually look out the window, that is, and aren't just clutching your table and chair as tightly as possible (really? did I do that?)

Originally, we were planning to take a boating cruise in the morning; there are several of them which circumnavigate Manhattan, and it seemed like a relaxing way to see a lot of New York City (from the water). But the cruise was cancelled and so we didn't go; in retrospect, this was probably to our benefit, as the weather that day was misty and with very low clouds, so much of the city would have been hidden in the haze.

Instead, we made our way down to Battery Park and Castle Clinton and took the ferry to Liberty Island and on to Ellis Island. Although the weather was indeed gray and misty, in a way this rather enhanced the trip, as Liberty Island emerged from the clouds to our great delight.

We didn't have the fancy tickets to climb up into the statue itself, so we contented ourselves with walking around the island and looking at the statue from ground level, which was quite enjoyable.

Then we returned to the ferry and proceeded on to Ellis Island. Although it doesn't have the Statue of Liberty on it, Ellis Island is in many ways a much more interesting place.

Over the last few decades, the Ellis Island facilities have been converted to an immense and extremely well-organized museum, telling the story of immigration and how it built the United States of America.

The main building is 3 massive stories tall, and nearly all of it is museum. Even though many of the exhibits are straightforward, and we made an effort to move through in a lively fashion, it was well over an hour to see what we saw, and I think we saw barely half of what there was to see.

The ferry returned us to Battery Park, and after some wandering around, and some lunch, the weather had cleared nicely, and we made our way up to Central Park.

I was very interested to see the John Lennon memorial in Central Park, and I wasn't disappointed. It is quite nice, and it was filled with people like myself, stopping to look and think a bit before moving on, all of us quietly part of a shared experience.

The weather was glorious, so we walked around Central Park for several hours. We moseyed across from Central Park West to Central Park East, stopping at places like the Bethesda Fountain, the Hans Christian Anderson statue, the Model Boat pond, and the Alice in Wonderland statue.

Every so often we would wander out of the park, but the surrounding areas weren't as nice, so we just kept wandering back into the park, walking up and down the tree-lined paths, marveling at all the different things to see.

After a while we were tired, so we found a nice spot on the Upper West Side to sit for a while and rest and talk; when we were restored it was already getting on to dusk, so we made our way down to Lincoln Center to see the fancy theaters.

Later we made our way out to Rockefeller Center, which was already all lit up for the holidays. We tarried for a while, watching the ice skaters in the ice rink, and wandering through the enormous Lego Store.

It was dinner time, and my plan had been to find one of the up-and-coming Indian restaurants in the so-called "Curry Hill" neighborhood near 28th and Lexington, but instead we ended up at a very nice spot a little bit farther north in Murray Hill, where we had a fine meal.

The next morning, we popped out of bed again and headed back downtown to the 9/11 Memorial. Although we were both well-acquainted with the events of 14 years ago, neither of us had been to Manhattan since, so we wanted to make a visit to the memorial part of our trip.

This is an extremely dramatic and moving place, obviously, and the memorial accomplishes its task(s) well, I thought. The overall presentation is quite remarkable: from the street-level entrance you make your way down, down, down. The farther you go, the more dramatic and powerful the experience becomes, until you reach the very bottom, where the bulk of the exhibits and memorial materials are located.

I was pleased to see that, for the most part, the memorial lets the facts speak for themselves, and focuses its attention on the people who were most directly affected: those in the towers, on the planes, and at the Pentagon, as well as the emergency personnel who responded to the events.

The displays are physical and immediate, incorporating objects from the buildings themselves (the stairs, the foundation columns, the slurry wall, the steel girders, etc.) as well as objects from the people involved (equipment, personal effects, etc.)

The memorial uses multi-media EXTREMELY effectively, playing actual clips from television broadcasts, 911 recordings, cell phone messages, interviews with witnesses and survivors, etc. A particularly dramatic and moving exhibit tells the remarkable (if by now quite well-known) story of Flight 93, moving back and forth between air traffic control recordings, voice mail messages, and other information to let the actual participants in the story tell it, speaking from the grave as it were in some cases. I glanced into that room for just a moment but was instantly captivated, and 10 minutes passed before I could breathe.

We hadn't expected to spend long at the memorial, but before we knew it we'd been there more than 2 hours, and had to drag ourselves away and on. Although upon leaving I felt like I hadn't really learned anything I didn't already know, I still felt like my visit was valuable and I don't regret going for an instant.

We both really enjoyed wandering around the various Manhattan neighborhoods, and I think we could have done much more of this if we'd had time. Some are rather straightforward, like walking through the Financial District or down the canyons of skyscrapers mid-town.

Others are still full of personality and character, like Greenwich Village, SoHo, the East Village, Murray Hill, or Chelsea. We got just ridiculously lost wandering around Greenwich Village: one of my personal goals had been to visit the Village Vanguard, but we ended up abandoning that quest and moving on; later, looking on the map, I realized that we had stopped in a falafel shop that was, quite literally, across the street from the Village Vanguard, and hadn't even known it.

New York is definitely quite expensive, and eating and drinking there was not cheap by any stretch. However, the food was remarkably good, much better than either of us had expected, full of fresh and good ingredients, well-prepared, well-presented, well-delivered. If I could afford it, I could easily spend all my time just wandering around Manhattan, eating and drinking and looking about...

Another very nice surprise was how successful we were at taking the subway all over the island, even with very little advance preparation and zero experience with the things that it often turns out you need to know about a city's transit system.

The hardest part of using the subway turned out to be finding the stations from above-ground. Once you were in the station, though, everything was well-marked and easy to find and in remarkably good condition given the astonishingly heavy use that the New York City subways receive.

Trains ran regularly, loudspeaker announcements and display signs were clear and accurate, the system as a whole seemed to be basically clean and safe, and all in all it was much better than I had anticipated.

That said, the subway was certainly not as nice as Seoul's subway, which is perhaps no surprise because Seoul's subway is brand new by comparison. As compared to the London Underground, though, I thought that the New York City subway system was at least as good, and certainly not as complex.

If you find yourself in La Guardia Airport, and want to get into midtown, the NYC Airporter is just fine, and certainly a bargain compared to airport-to-downtown options I've used elsewhere.

If you want a place to stay, and are looking to be part of everything, the Marriott Marquis is right smack in Times Square, in the middle of the action, but once you duck into your room and close the door, it's peaceful and welcoming as can be. And boy is it convenient to get to anywhere else in Manhattan from that location!

Oh, and my wife got to go see Kinky Boots at the Hirschfeld on Saturday night, but you'll have to ask her about that (I was in Port Chester at the time, as we've already discussed).

Categories: FLOSS Project Planets

Justin Mason: Links for 2015-11-10

Planet Apache - Tue, 2015-11-10 18:58
  • No Harm, No Fowl: Chicken Farm Inappropriate Choice for Data Disposal

    That’s a lesson that Spruce Manor Special Care Home in Saskatchewan had to learn the hard way (as surprising as that might sound). As a trustee with custody of personal health information, Spruce Manor was required under section 17(2) of the Saskatchewan Health Information Protection Act to dispose of its patient records in a way that protected patient privacy. So, when Spruce Manor chose a chicken farm for the job, it found itself the subject of an investigation by the Saskatchewan Information and Privacy Commissioner.  In what is probably one of the least surprising findings ever, the commissioner wrote in his final report that “I recommend that Spruce Manor […] no longer use [a] chicken farm to destroy records”, and then for good measure added “I find using a chicken farm to destroy records unacceptable.”

    (tags: data law privacy funny chickens farming via:pinboard data-protection health medical-records)

Categories: FLOSS Project Planets

Edward J. Yoon: Cut&amp;Paste from quora

Planet Apache - Mon, 2015-11-09 23:07
텐서플로우라는 오픈소스가 핫하다. 내실이 부족하니 더더욱 외부 뉴스에 흔들흔들.

차마 내가 못한 얘기를 Microsoft 직원이 시원하게 싸질러주었네. 요약하면, 잃는건 적고 얻는게 크다는 것. 우리도 쓰게 마음껏 재주부려봐라.

The open field of deep learning frameworks is already a crowded and hyper competitive. Google doesn't gain much by keeping its technology, which may be the best but not far superior than other similar state-of-the-art frameworks such as Theano/Torch/CNTK/minerva/etc., behind a closed door. If you read their white paper and other tutorials/comparisons of Theano/Torch/CNTK/minerva, you would easily develop a headache of reading the same concepts over and over again. All are open source and are quick to copy/implement the greatest from each other. Google got the most descriptive name but all these software are about expressing a tensor graph in a high-level language, which gets translated to high performance code which can be run on (multi) CPUs/GPUs. 

Google's direct competitors in the AI space are already leading other deep learning frameworks. So even if TF is superior, it will be very slow painful experience if these competitors decide to jump ship.

So let's keep in mind that their loss for publishing yet-another-deep-learning-framework is small. However, the benefits are tremendous.

  • Due to the competitive nature of deep learning frameworks, owning yet-another-state-of-the-art framework doesn't set AI companies apart. What make a company outstanding are services built on top of this framework and the people. On the software aspect, I believe that Google Brain still has secret weapons built on top of TF. On the people aspect, by releasing this software, Google excites not just the research community but also their own army and the potential hires.
  • At Google size, you cannot take for granted that other teams/individuals would automatically use your product. You need to shout out very loud (and to the public) that your system is the best to gain other team's confidence, which helps create synergy and increases the company productivity.
  • Obviously, by open sourcing, they get the benefits from contributors outside the core team.
  • Once TF gets popular, many new research ideas would be implemented in TF first, which makes it more efficient for Google to productize those ideas and have advantages over competitors.
  • TF helps increase the credibility of future Google's research paper. Google research is notorious for not publishing their code.
  • And last but not least, open sourcing is fun and rewarding. It increases the morale of the core team too.
Categories: FLOSS Project Planets

Justin Mason: Links for 2015-11-09

Planet Apache - Mon, 2015-11-09 18:58
  • Caffeine cache adopts Window TinyLfu eviction policy

    ‘Caffeine is a Java 8 rewrite of Guava’s cache. In this version we focused on improving the hit rate by evaluating alternatives to the classic least-recenty-used (LRU) eviction policy. In collaboration with researchers at Israel’s Technion, we developed a new algorithm that matches or exceeds the hit rate of the best alternatives (ARC, LIRS). A paper of our work is being prepared for publication.’ Specifically:

    W-TinyLfu uses a small admission LRU that evicts to a large Segmented LRU if accepted by the TinyLfu admission policy. TinyLfu relies on a frequency sketch to probabilistically estimate the historic usage of an entry. The window allows the policy to have a high hit rate when entries exhibit a high temporal / low frequency access pattern which would otherwise be rejected. The configuration enables the cache to estimate the frequency and recency of an entry with low overhead. This implementation uses a 4-bit CountMinSketch, growing at 8 bytes per cache entry to be accurate. Unlike ARC and LIRS, this policy does not retain non-resident keys.

    (tags: tinylfu caches caching cache-eviction java8 guava caffeine lru count-min sketching algorithms)

  • What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability.

    The ever-shitty Java serialization creates a security hole

    (tags: java serialization security exploits jenkins)

  • Gallery – Steffen Dam

    Danish glassware artist making wonderful Wunderkammers — cabinets of curiosities — entirely from glass. Seeing as one of his works sold for UKP50,000 last year, I suspect these are a bit out of my league, sadly

    (tags: art glassware steffen-dam wunderkammers museums)

  • London garden bridge users to have mobile phone signals tracked

    If it goes ahead, people’s progress across the structure would be tracked by monitors detecting the Wi-Fi signals from their phones, which show up the device’s Mac address, or unique identifying code. The Garden Bridge Trust says it will not store any of this data and is only tracking phones to count numbers and prevent overcrowding.

    (tags: london surveillance mobile-phones mac-trackers tracking)

  • Red lines and no-go zones – the coming surveillance debate

    The Anderson Report to the House of Lords in the UK on RIPA introduces a concept of a “red line”:

    “Firm limits must also be written into the law: not merely safeguards, but red lines that may not be crossed.” …    “Some might find comfort in a world in which our every interaction and movement could be recorded, viewed in real time and indefinitely retained for possible future use by the authorities. Crime fighting, security, safety or public health justifications are never hard to find.” [13.19]  The Report then gives examples, such as a perpetual video feed from every room in every house, the police undertaking to view the record only on receipt of a complaint; blanket drone-based surveillance; licensed service providers, required as a condition of the licence to retain within the jurisdiction a complete plain-text version of every communication to be made available to the authorities on request; a constant data feed from vehicles, domestic appliances and health-monitoring personal devices; fitting of facial recognition software to every CCTV camera and the insertion of a location-tracking chip under every individual’s skin. It goes on: “The impact of such powers on the innocent could be mitigated by the usual apparatus of safeguards, regulators and Codes of Practice. But a country constructed on such a basis would surely be intolerable to many of its inhabitants. A state that enjoyed all those powers would be truly totalitarian, even if the authorities had the best interests of its people at heart.” [13.20] …   “The crucial objection is that of principle. Such a society would have gone beyond Bentham’s Panopticon (whose inmates did not know they were being watched) into a world where constant surveillance was a certainty and quiescence the inevitable result. There must surely come a point (though it comes at different places for different people) where the escalation of intrusive powers becomes too high a price to pay for a safer and more law abiding environment.” [13.21]

    (tags: panopticon jeremy-bentham law uk dripa ripa surveillance spying police drones facial-recognition future tracking cctv crime)

  • Dublin is a medium-density city

    Comparable to Copenhagen or Amsterdam, albeit without sufficient cycling/public-transport infrastructural investment

    (tags: infrastructure density housing dublin ireland cities travel commuting cycling)

Categories: FLOSS Project Planets
Syndicate content