Feeds

These are the things that are likely to happen:

1. A new cpan(1) (the thing from App::cpan and also bundled with CPAN.pm) version for Perl 5.14. I need to get on that pretty soon, then. :) The current cpan(1) has support for loading alternate config files, and I'd like to also provide a config option that acts as much as possible like cpanm. I think that's probably 90% possible, even if we have to convince CPAN.pm to do things slightly separately.

2. A new edition of Learning Perl. I've been slowly working on that for awhile and keeping track of myself at a blog I use to track its progress. This edition will cover up to Perl 5.14. Our goal is dead trees by OSCON. There will also be an update to the Learning Perl Student Workbook.

3. A new edition of Intermediate Perl. We're going to drastically update the distribution creation section (not for Dist::Zilla, though, probably, because we're constrained to teaching people how to do things without relying on a tool), some very light Moose stuff, and some better testing stuff. I also want to add a chapter just for regex objects. Most of that is already written, but it's time to let other people see it. Our goal is dead trees by OSCON, just like Learning Perl.

4. Update a couple of chapters of Mastering Perl. This book probably won't see another dead trees edition, but O'Reilly is game to try some other things, like updates in Safari or PDF updates.

5. See what Dist::Zilla things I might steal for Module::Release. I'm not a fan of Dist::Zilla only because I don't like any tool that forces me to use that tool, but that are probably some good ideas I can steal anyway.

6. I've indexed most of BackPAN. Now it's time to make it easier to get and use the data. Some of this goes back to the PAUSE web service that I started at the 2010 Vienna QA workshop.

7. Keep going with The Effective Perler, the companion website to Effective Perl Programming. We had at least one meaty Perl Item every week, and I might be able to pull that off again.

I haven't thought too much about my conference schedule this year, but I'm interested in spending my conference time budget on some new events or new locations. There are so many Perl events now that I have to stop feeling guilty about picking and choosing. Instead of doing the same events I've done in the past, I want to mostly go to events I haven't attended yet. I need to lighten up the summer too. "Conference season" has become too much of a black hole. I need to reclaim some of that time to get real work done, so I have to pick and choose in June-July-August so I'm not traveling or recovering from traveling for the entire summer.

I've also had to concede that I need to make my conference activities cash neutral. I don't need to come back with more money that I started with, but I can't lose any. I try to do that by teaching classes, whether for the public or for sponsors. I want to work for my sponsorship. That's getting a bit harder since more people are doing that though. :)

• Frozen Perl in Minneapolis. I'll be teaching Effective Perl Programming the Friday before, and giving the keynote. I've been there before, but it's so close, and it reminds me that it's really not that cold in Chicago (it was -15 C in Minneapolis when I was there on New Years Day).
• OSCON, which I've done many times, but mostly because I'm on the conference committee and a lot of important people from my primary publisher show up.
• São Paulo Perl Workshop—I'd never been south of the equator before 2010, but I had separate trips to Buenos Aires and Santiago. Now I want to visit Brazil. Randal has been down there and says they have conference sizes that put almost every other event to shame.
• YAPC::Asia—I really enjoyed my trip to Tokyo and Shibuya.pm, so I'd like to try that again. I don't think a class would work out there unless I learn Japanese very quickly. :)
• London Perl Workshop, because I haven't been to London for awhile and I hear it's really nice in December.
• YAPC::EU in Riga, I hope, but it's really close to OSCON. I have to be in Europe around then anyway, so I think it's a matter of scheduling and figuring out how to cover the costs.
• OSDC::AU - There's a solar eclipse with prime viewing in Australia on November 14. If I can stretch out the trip a bit, I might be able to do both.
• I'll have to check the QA Workshop news because I haven't paid attention to that, but good things always happen there.

Happy New Year everyone!

Here it is, only a few hours into the new year, and I'm already breaking a resolution of sorts. It wasn't a proper resolution, but given that I have plenty of things on my plate — not the least of which is a baby on the way — I had decided that I wasn't going to hack on Role::Basic for a bit until I saw the dust settling.

First, a simple question: if you're interested in Role::Basic, is it as a stepping stone to Moose, or because you just want roles and nothing else? The answer to this question could have a huge bearing on a fundamental problem that I face. This post is going to be rather long, so if all you do is answer that question, that's fine. I need to know. The reason this post is going to be long is because making things hard is easy. Making things easy is hard. Thus, in trying to solve a problem in an "easy" way, I have to think about a hard problem.

The problem with waiting for the Role::Basic dust to settle is that there's a touch more dust than I thought. Via private email, Facebook and Twitter, I already have several people thanking me and telling me they're going to start using this module. It could work its way into my $work, there's already a module on the CPAN which lists Role::Basic as a dependency and though it only has 11 watchers, the Role::Basic github repository is already my most watched module (outside of HOP which is merely a port of Dominus' code). On top of that, Matt Trout has opened a bug report against Role::Basic, agreeing with its goals, but concerned it will cause confusion.

To add to the fuss, it took only three days after the initial release for a complaint about the same problem the original traits researchers hit, I need to properly handle state. That's where the real problem kicks in.

Traits and No State

The original traits paper stated:

Traits do not specify any state variables, and the methods provided by traits never directly access state variables.

Reading further into the paper, it makes it clear that traits are allowed to access state via getters and setters and these are added to the list of methods the trait requires but the trait must never define any state of their own. The reasoning behind this was simple: if traits could define their own state and don't know about other traits, you could still have two or more traits attempting to define the same state and effectively get you back to the sort of conflicts that multiple inheritance could cause (this might seem strange to Perl 5 programmers, but it's as if two roles reached inside of a blessed hashref to diddle the same "_counter" slot).

Thus, in the original traits model, we the following:

class = state + traits + glue

Since traits were not allowed to manipulate state variables directly, any manipulation went into that "glue". If a trait wanted its own state, all it could do is list methods which it required and have the class implement them in that "glue". Thus, Marcel Grünauer had a problem because some of his traits, ahem, "roles", wanted state, but he couldn't do this:

package My::Role; use Class::Accessor::Lite rw => [qw(foo bar)]; use Role::Basic;

This is because the role has no way of knowing if methods defined outside of the role are "helper functions" or getters/setters. The role only provides methods defined in that role or defined in another role the first role consumes. According to the original traits model, the "proper" solution to Marcel's problem is the following:

package My::Role; use Role::Basic; requires qw(foo bar);

And in the consuming class:

package My::Class; use Role::Basic 'with'; # this is the infamous "glue" code we're talking about it use Class::Accessor::Lite rw => [qw(foo bar)]; with 'My::Role';

That works, but there are a couple of issues with this. It turns out that not allowing traits to define state was a difficult constraint and multiple ways of implementing stateful traits have been proposed. This was largely because programmers were getting frustrated at having to duplicate the glue in every class which used a given role. Weren't roles supposed to reduce that duplication? And if the role requires another accessor, now we have to go into every class consuming that role and add it to the "glue". Ugh! This is a sticky problem.

This also means that we're introducing some scaffolding that will mean more code changes will be necessary if/when you wish to upgrade from Role::Basic to Moose.

A Prelude to Moose?

This all leads to my wondering if the interest expressed is just for roles or a prelude to Moose. If this module is a "prelude to Moose", then I should strive to make the upgrade path as easy as possible. Should I provide a "has" subroutine so you can declare attributes? But if I do that, you need to be able to set them in the constructor which implies I should provide a default constructor which implies that creeping featuritus has struck and I'm in danger of working on reimplementing Moo or Mouse. This is bad. I just wanted to make it easy to use roles.

If, however, people want "just roles" and not a prelude to Moose, then some artificial constraints could go away. For example, in Matt Trout's bug report, he mentioned:

Additionally, doesn't -excludes entirely exclude the method? Don't we actually want a form that says "I am going to provide my own version so convert it into a requires"?

I agree with Matt on this one. I had brought this issue up about a year and a half ago,¹ but I think it got lost in the wake of other discussions. However, I deliberately did not introduce this feature because Moose doesn't implement it. The following on my machine prints 1.21:

{ package Some::Role; use Moose::Role; sub foobar { print "hi!" } } { package Some::Class; use Moose; with 'Some::Role' => { -excludes => 'foobar' }; } my $o = Some::Class->new; print Moose->VERSION;

If you later try to call the "foobar" method, you will get the standard "Can't locate object method" runtime error instead of a composition time error. This should also apply to the '-alias' feature because of this:

if ( $object->DOES('Some::Role') ) { # you've promised that the object does this role # the consumer has to assume that the object # provides the role's behavior }

I actually think I can implement this and it won't break the upgrade path since we're being more restrictive rather than more permissive, but I want to think very carefully about this.

If I go the "roles only" route, I can have greater flexibility, but will need to evaluate everything on whether or not the expected benefit is greater than the cost of making it harder to convert to Moose. However, this seems like a bad idea if people are using this module as bait to convince others (or themselves) that shacking up with a Moose is a good idea.

In the meantime, this still doesn't give me a great solution to Marcel's problem. Rats!

1. I now see that I filed a bug about adding excluded methods to requirements, but it was considered to be a non-trivial problem and the ticket is still open.

• The 2011 Cricket World Cup: A Documentary : my mate Sush is looking to fund ‘a documentary set in India during the World Cup of Cricket in 2011 about Indian cricket fans and their personal stories.’ Looks great — might blog about this a bit more…
  (tags: sush movies kickstarter funding documentaries cricket india)

• on URL Design : from one of GitHub’s designers, good tips on how the URL UI needs to work these days
  (tags: github urls design ui usability webdev webdesign http)

• 27C3: Console Hacking 2010 : great preso on the PS3 hack from the fail0verflow team. love the LaTeX “science bit”. Sony’s epic fail: non-random “random” key data
  (tags: ps3 hacks console crypto hypervisor security ccc fail0verflow)

Yesterday we had our first revived Tel Aviv Perl Mongers meeting. You probably came across the announcement once or twice in your RSS feeds, mailing lists or even through a personal message from me.

Apparently this worked quite well. The last Perl Mongers meeting included 5 to 6 times as many people as previous meetings. It was pretty awesome. We also scored pretty high on the variety of people.

I got to meet a lot of new people, hear interesting talks and mainly have a lot of laughs.

After the talks, the top-posters went out to dinner and had a great time. The discussion of how to represent a salad in pure OO form ensued and all hell broke loose from then on! :)

Next meeting we're hoping to get even more people to attend. My plans include shorter talks and a lot more fun (such as lightening talks) and getting more people to step up (help organize and give presentations on various issues).

Thanks to everyone who came, everyone who gave talks, helped with the website (DNS, hosting), graphics, flyers, advertising and so on.

See you next time!

As predicted in the October Summary, the latest milestone for CPAN Testers came just before Christmas. On 22nd December to be exact, as can be seen on the Interesting Stats page of the CPAN Testers Statistics site. Once again, many thanks to all the testers who have help to contribute to the milestone.

Congratulations to Chris Williams for posting the 10 millionth report. It was a PASS for App-cpanminus-1.1005.

Cross-posted from the CPAN Testers Blog.

• One of the ICE domain seizures was a legit mp3 blog, posting legal promo mp3s : At least one of the sites seized by DHS was an mp3 blog which posted authorised, promotional mp3s, sent from record label VPs and artists — ie. none of the supposedly “infringing” files, actually were infringing. (via Tony Finch)
  (tags: mp3 music piracy law ice dhs filesharing copyright copyfight techdirt via:fanf seizure mp3blogs for:nialler9)

I haven't done much with Perl this week. I added CPAN::Meta::YAML to the perl core, but otherwise spent the week with family and friends and digging out from the great East Coast blizzard of 2010. I have a lot of toolchain module items to work on and hope to have more to say about that by the New Year.

A long time ago I posted about Roles without Moose and while I still feel that for most cases Moose is the way to go, there can still be a bit of resistance to the idea. Matt Trout responded to my post with how one could have just roles (read his entire post to understand the context):

package Foo::Manual; use Moose; extends 'UNIVERSAL'; # get rid of Moose::Object with 'Foo::Manual::Bar'; sub new { bless {} => shift }

This still involves putting Moose on your servers and when you're faced with a large dev team that is very conservative in their approach, this might be an uphill battle. So what are my alternatives?

You could grab Mouse and rename Mouse::Tiny and get roles, but one of the first questions I was asked was "wasn't that abandoned?" Well, it's been picked up again. And you still have a whole bunch of other things in there which make ultra-conservative devs twitch.

So I was pretty happy to see Role::Tiny released with Moo. The idea was that you could have just roles, in a single package, and nothing else. This makes it easier convince others to dip their toes in the water and yank them out again if they're not comfortable. Of course, at the end of the day, it's not a crime to want roles without all of the extra bits, no matter how shiny those bits may be.

Alas, Role::Tiny had a couple of issues (not the fault of the author). First, here's how you use more than one role:

with 'Some::Role1'; with 'Some::Role2';

Thus, you lose compositional safety and these are not much better than mixins. When I asked Matt Trout about this on Twitter, Frew Schmidt replied:

@OvidPerl @shadowcat_mst when I asked him on IRC a week or so ago he said write tests and he'd do it. So I guess tag your it :-)

That made me very happy and I was going to start, but then I noticed a couple of things. First, Role::Tiny is bundled with Moo. The more modules I have to convince someone to install, the harder it will be to get them installed. Perhaps Role::Tiny will have its own distribution in the future, but then I noticed that it has the before, around and after method modifiers. This is problematic to me.

I've seen people (myself included) using these method modifiers rather casually. Unfortunately, if these modifiers have any side-effects then the order in which they are composed becomes important. For example, what does the following code print?

#!/usr/bin/env perl { package Some::Role; use Moose::Role; requires qw(some_method); before some_method => sub { my $self = shift; $self->some_number( $self->some_number + 2 ); }; } { package Another::Role; use Moose::Role; requires qw(some_method); before some_method => sub { my $self = shift; $self->some_number( $self->some_number / 2 ); }; } { package Some::Class; use Moose; my @roles = int( rand(2) ) ? qw(Another::Role Some::Role) : qw(Some::Role Another::Role); with @roles; has some_number => ( is => 'rw', isa => 'Num' ); sub some_method { print shift->some_number, $/ } } my $o = Some::Class->new( { some_number => 7 } ); $o->some_method;

That will print either '4.5' or '5.5', depending on the order in which the roles are consumed. This violates part of the original intent of roles (traits) to be a declarative way of safely decoupling class responsibility and shared behavior. One of the issues with inheritance that roles were designed to fix is the ordering issue (MRO shouldn't be a problem) and I was concerned that introducing method modifiers to devs new to roles might open up a new can of worms. An experienced developer should remember "modifiers should not modify state" (you're forgiven if that's confusing), but the more "rules of thumb" you have to remember, the more "rules of thumb" you'll forget.

Thus, Role::Basic is now on github (and will likely go to the CPAN soon). Here's the core idea:

Here's how to use it.

In a role:

package Does::Serialize::AsYAML; use Role::Basic; use YAML::Syck; requires 'as_hash'; sub serialize { my $self = shift; return Dump( $self->as_hash ); } 1;

In your class:

package My::Class; use Role::Basic 'with'; with qw( Does::Serialize::AsYAML ); sub as_hash { ... } # because the role requires it

Nothing fancy. Nothing strange. It's also designed to have a Moose-like syntax to make it easy to migrate to Moose later. Aside from the Moose-like syntax, the primary design goals are safety and simplicity, while still holding true to the spirit of roles. Here's what's come out of this:

• Basic role support including composing into your class, composing roles from other roles, roles declaring requirements, and conflict resolution via aliasing and exclusion.
• Moose-like syntax
• No handling of the SUPER:: bug
• Composition safety (you can't call with() more than once)
• Override safety (optional support for noticing when a class silently ignores a role method)
• No instance application (this might change)
• No method modifiers

If you need something which this module does not provide, you are strongly encouraged to look at Moose or other options.

In Mason 1 there are several ways to filter content: the <%filter> section, escape flags, and Component Calls with Content. Each of these has their own disparate implementation and API.

In Mason 2 we aim to unify these with a single implementation and a more consistent syntax.

A set of standard filters are automatically available in components, and other filter packages can be loaded via plugins.

Invoking filters (block syntax)

Here’s the typical way of invoking a filter:

<% $.Trim { %> This string will be trimmed </%>

Things to note here:

• An { at the end of a <% %> tag denotes a filter call.
• The </%> tag marks the end of the filtered content.
• The expression $.Trim, aka $self->Trim, is a method call on the component object which returns a filter. In general everything before the brace is evaluated and is expected to return a filter or list of filters.
• Filter names use CamelCase to distinguish themselves from other methods in the Mason::Component namespace. Another option was to use a standard prefix or suffix, e.g. trim_filter or filter_trim, but I thought this was the least ugly option and made filters look a little more like built-in tags.

Filters can take arguments:

<% $.Repeat(3) { %> There's no place like home. </%> ==> There's no place like home. There's no place like home. There's no place like home.

Again, the expression $.Repeat(3) returns a filter, meaning that it can
be curried:

% my $repeat_three = $.Repeat(3); <% $repeat_three { %> There's no place like home. </%>

A simple filter is just a subroutine that takes text as input and return the
new text. Thus you can create one-off filters with anonymous subroutines:

<% sub { reverse($_[0]) } %>Hello</%> ==> olleH

Filters can be nested, with separate tags:

<% $.Trim { %> <% sub { uc($_[0]) } { %> This string will be trimmed and uppercased </%> </%>

or within a single tag:

<% $.Trim, sub { uc($_[0]) } { %> This will be trimmed and uppercased </%>

Multiple filters within the same tag are applied, intuitively, in reverse order with the last one being innermost. e.g. in this block

% my $i = 1; <% $.Repeat(3), $.Cache($key, '1 hour') { %> <% $i++ %> </%> => 1 1 1

the output of <% $i++ %> is cached, and then repeated three times, whereas in this block

% my $i = 1; <% $.Cache($key, '1 hour'), $.Repeat(3) { %> <% $i++ %> </%> => 1 2 3

<% $i++ %> is executed and output three times, and then the whole thing cached.

Invoking filters (pipe syntax)

Filters can also appear in a limited way inside a regular <% %> tag; this replaces Mason 1’s escape flags.

<% $content | NoBlankLines,Trim %>

The filter list contains one or more comma-separated names, which are treated as methods on the current component class. With this syntax you cannot use anonymous subroutines or variables as filters, or pass arguments to filters (but you can define local filter methods to get around this).

One common use of this form is to escape HTML strings:

<% $message_body | H %>

The H filter is provided along with other web-related filters in Mason::Plugin::HTMLFilters.

Applying filter to current component

Mason 1’s <%filter> section was good for filtering the content of the current component. e.g. to make it uppercase:

<%filter> $_ = uc($_); </%filter>

In Mason 2 the most succinct replacement is currently an around modifier:

<%around main> <% sub { uc($_[0]) } { %> % $self->$orig(); </%> </%around> Creating filters

Here’s a filter package that implements two filters, Upper and Lower:

package MyApp::Filters; use Method::Signatures::Simple; use Moose::Role; method Upper () { return sub { uc($_[0]) } } method Lower () { return sub { lc($_[0]) } } 1;

To use these in a component:

<%class> with 'MyApp::Filters'; </%class> <% $.Upper { %> ... </%>

Or if you want them available to all components, put them in Base.pm at the top of your component hierarchy, or in your application’s Mason::Component subclass.

Simple vs. dynamic filters

A simple filter is a code ref which takes a string and returns the output, like Upper and Lower above.

A dynamic filter (a Mason::DynamicFilter object) contains a code ref which takes a yield block and returns the output. A yield block is a zero-argument code ref that returns a content string.

The power of dynamic filters is that they can choose if and when to execute the yield block. For example, here’s the standard Cache filter in action:

<% $.Cache($key, '1 hour') { %> ... some computed content ... </%>

This caches the inner content for one hour based on cache key $key. Here’s the implementation:

method Cache ( $key, $set_options ) { Mason::DynamicFilter->new( filter => sub { my $yield = shift; $self->cmeta->cache->compute( $key, $yield, $set_options ); } ); }

Using CHI’s compute method, we check the cache first and return the output immediately if it is available. Only on a cache miss do we actually execute the (presumably expensive) yield block. This could not be implemented with a simple filter since the content would be computed every time.

yacc was a major breakthrough. For the first time, automatic generation of of efficient, production-quality parsers was possible for languages of practical interest. Yacc-generated parsers had reasonable memory footprints. They ran in linear time.

But error reporting was overlooked. Then as now, the focus in analyzing algorithms was on power -- what kinds of grammar an algorithm can parse -- and on resource consumption. This leaves out something big.

Our frameworks for analyzing things affect what we believe. We find it hard to recognize a problem if our framework makes us unable to articulate it. Complaints about yacc tended to be kept to oneself. But while yacc's overt reputation flourished, programmers were undergoing an almost Pavlovian conditioning against it -- a conditioning through pain.

With yacc, you no longer need to write your own parser. If you put your grammar into a form that yacc will accept, yacc writes your parser for you. But over the years, people noticed -- it usually takes longer to put a grammar into a form that yacc will accept than it does to write a recursive descent parser from scratch. This is almost always true when someone was using yacc for the first time. But it is usually true for yacc experts as well.

Certain tools do have a high initial cost. They make this acceptable with a payoff over the lifetime of the software. But in this respect also, yacc does worse than hand-written recursive descent -- much worse.

The structure of a recursive descent parser reflects the structure of the language being parsed. Small changes in the language tend to require only small changes in the parser. Major changes usually affect only that part of the grammar actually changed. And how the recursive descent parser must change, and why, is usually obvious even to a programmer who does not make a specialty of parsing.

For a yacc grammar, the change process is not incremental -- each iteration is almost like starting from scratch. yacc parsers work using LALR automata. Small changes in the grammar often cause big changes to the LALR states. Tracing the logic behind these changes is a challenge even to those familiar with the underlying math -- one which they usually find not worth the time and effort. This is one reason that those experienced with yacc find it nearly as hard to use as beginners did -- for real-world problems, understanding the LALR states is simply too hard for anyone. Experts, like non-experts, are forced to fix yacc grammars using trial and error.

yacc's demands follow the nearly incomprehensible logic of the LALR automaton. Adapting a grammar to yacc is a struggle, during which you watch your grammar become less and less a reflection of what you were originally trying to do. One of the most difficult tasks in programming, it is almost one of the most unsatisifying and unrewarding.

The Fourth Requirement for Replacing yacc: Easy Diagnosis of Grammar Problems

The easiest way to deal with grammar problems is to arrange for them not to happen. You can do this if you have a notation for the grammar which is

1. A natural and intuitive way to express the grammar, and which
2. Makes it literally impossible to specify a problem grammar.

The notation for regular expressions has these two properties, and that is a major reason for the enduring popularity of regular expressions. Once you get used to its limits, regular expression notation becomes a natural way to describe languages. And regular expression notation makes it impossible to specify anything that is not a regular expression.

More powerful parsing algorithms have these same two properties when they accept all context-free grammars (as Marpa does). The context-free grammars are those you can write in BNF, and vice versa. BNF is (again, modulo some getting-used-to) a natural and intuitive way to express a language, and you don't have to worry about specifying a language which is harder than context-free -- the BNF notation makes that impossible.

yacc uses BNF as its notation for expressing grammars, but the most natural way to express a practical grammar in BNF is almost never an LALR grammar. No natural or intutive notation is known that describes only LALR grammars, even after 40 years of considerable interest in them. I have to doubt that such a notation ever will be found.

Notes

Note 1: It has definitely been the tradition to understate the importance of error reporting, or even to ignore it entirely. But I should point out that I have not consulted Knuth's and DeRemer's original papers, which are behind paywalls. Also, things seem to be getting better, particularly with the arrival of the very important textbook by Grune and Jacobs, which does devote significant attention to the error reporting properties of the various algorithms.

Note 2: Perhaps because reporting that you found it impossible to use one of the standard tools in our field was more likely to produce conclusions about your competence than about the tool, the tradition among yacc users was to suffer in silence. One good 1995 account of trials with yacc is this contribution to comp.compilers. And there is one person who I believe has an intuitive understanding of LALR: Larry Wall. Certainly I doubt there is anyone alive whose practical knowledge of LALR is better than Larry's. That makes it very significant that Perl 6 does not use yacc or any other LALR based parser.

Note 3: Here I am talking about pure regular expressions. Perl regexes are much more powerful than regular expressions, and the power comes with tradeoffs: The notation is no longer as simple or intuitive.

Merry Christmas!

It fills me with great joy to finally announce the release of Mojolicious 1.0 (Snowflake).
Many thanks to everyone who helped making it happen!

This event also brings back a lot of memories from 2005, when I unveiled its predecessor Catalyst for the first time, shortly before the Ajax revolution.

Right now we are once again at the verge of a similar revolution happening with HTML5.
Browsers are getting much smarter and the server side thinner.

There are entirely new challenges to tackle, like persistent bi-directional communication through WebSockets, offline-enabled web applications and tighter integration with JavaScript.

The Web is in a constant evolutionary state, browsers like Google Chrome gain new features almost daily.

Perl itself is changing too with Perl 6 no longer being just a pipe dream, you have to anticipate a migration at some point.

These are uncertainties that require some rethinking.
By making Mojolicious a mostly self contained system of small building blocks we gain the necessary flexibility to ensure longevity for the project and a great user experience at the same time.

Having no prerequisites besides Perl 5.8.7 is of course not a rule, and in fact we do use quite a few optional CPAN modules to provide advanced functionality, but making the installation process this painless turned out to be a huge success with our early adopters.

The innovation doesn't stop there.
We have developed an entirely new workflow for web application development in Perl, formalizing the growing process from single file prototype to well structured MVC web application.

Fun oneliners scraping the latest news from Reddit are just as easy as zero downtime software upgrades with the built in production HTTP server.
When we say "web framework" we actually mean it. :)

The project is very well documented thanks to a Perl Foundation grant, you should take a look at it on our brand new website at mojolicio.us.

Have fun!

Permalink | Leave a comment  »

This Wednesday (Dec. 29th) we'll have a TA.pm meeting of the Tel Aviv area (and anyone who wants to come visit!) at Shenkar College in Ramat Gan.

If you're interested in Perl (to learn, to improve, to steal cool stuff, to meet new interesting people), this meeting is for you!

PNG flyer.
PDF invitation.

My work on a TPF grant to improve documentation and test coverage of the Parrot Embedd API is going well. I have added extensive examples of Parrot function and type signatures, as well beginning to increase test coverage of this subsystem.

I am working in the leto/embed_grant branch on Github, which has already been merged to master just before the release of Parrot 2.11.0.

My first merge of this topic branch included about 15 commits, which are about 2/3rds documentation and 1/3rd tests. I clarified some points about edge cases of Parrot function signatures, such as void input and void output, which is the emtpy string concatenated to both sides of an arrow ->, and gave an expanded description of what Pi means (PMC, invocant). Many examples of diverse kinds of function signatures were also added.

I gave the first user-visible documentation for many constants in the Embed subsystem, such as debug flags when creating interpreter objects and inline descriptions of different runcores that can be used with interpreter objects.

The tests that I added include the first coverage of returning Float PMCs and numeric constants from our embedding subsytem, as well as additional coverage for returning a ResizablePMCArray consisting of Numeric PMCs. I also added tests for creating multiple interpreter objects and added a TODO test for Trac Ticket 1880.

I also fixed a bug in the Parrot test suite, where tests in t/src were not skipped properly if src/parrot_config.o did not exist.

Most of this work was done between Thanksgiving and holiday travel, so I expect that development pace will pick up in the next few days. Currently, one of four inchstones has been achieved, and I will concentrate on raising the code coverage of extend_vtable.c in the next two weeks.

I would like to thank The Perl Foundation, and Karen Pauley in particular, for funding this very important grant to the Parrot and Rakudo Perl 6 communities.

Recently, I needed to clone a distribution on CPAN and release it under a new name. I also need to keep it in sync with the original distribution. Rather than do any of that by hand, I decided to whip up a Dist::Zilla plugin (two, actually) to do the job for me.

The background for this particular situation involves the role that YAML plays in the Perl 5 module toolchain. For many years, now, CPAN distributions have included META.yml files containing distribution metadata. For CPAN clients to read this metadata, the Parse::CPAN::Meta module (P::C::M) was added to the Perl 5 core in Perl 5.10.1. P::C::M is (unfortunately) a read-only copy of YAML::Tiny.

The situation became more complex after the Oslo Consensus, which agreed on using the META.yml format as a standard way for module toolchain components to communicate post-configuration dependencies. This means module builders like Module::Build and ExtUtils::MakeMaker need to be able to write metadata files, not just read them. Thus, Module::Build::YAML was added, which is a full-fledged copy of YAML::Tiny that allows Module::Build to bootstrap if YAML::Tiny is not installed. Likewise, ExtUtils::MakeMaker needs ExtUtils::MakeMaker::YAML for its own bootstrap.

Thus, we seemed destined for three copies of YAML::TIny of various degrees of fidelity to be tucked away in the Perl 5 core under alternate names.

This situation is, frankly, nuts. Fortunately, the Perl 5 pumpking agrees.

Thus, it seemed that YAML::Tiny was destined for the Perl 5 core, instead. But a wrinkle emerged. YAML::Tiny is not really YAML, but only a subset. It is sufficient for META.yml files, but whatever goes is core to support that should not be considered a 'preferred' YAML library. A compromise was struck, to call it CPAN::Meta::YAML instead.

Now, I needed to create CPAN::Meta::YAML and make it easy to track YAML::Tiny.

The first Dist::Zilla plugin I wrote was Doppelgaenger, which downloads the latest stable release of a module and uses it as the basis for a new, renamed distribution. It also strips the Pod and version number from the source. The second plugin is AppendExternalData, which I use to add new Pod that is specific to CPAN::Meta::YAML.

The final CPAN::Meta::YAML repository contains only four files:

Changes dist.ini pod/lib/CPAN/Meta/YAML.pm MANIFEST.SKIP

And here is the dist.ini file:

name = CPAN-Meta-YAML author = Adam Kennedy <adamk@cpan.org> author = David Golden <dagolden@cpan.org> license = Perl_5 copyright_holder = Adam Kennedy copyright_year = 2010 [Doppelgaenger] source_module = YAML::Tiny strip_pod = 1 strip_version = 1 [AppendExternalData] source_dir = pod [MetaResources] ;repository.url = git://github.com/dagolden/cpan-meta-yaml.git ;repository.web = http://github.com/dagolden/cpan-meta-yaml/ ;bugtracker.web = http://rt.cpan.org/NoAuth/Bugs.html?Dist=CPAN-Meta-YAML [@Filter] -bundle = @DAGOLDEN -remove = Prepender -remove = PodCoverageTests -remove = PortabilityTests git_remote = github [RemovePrereqs] remove = YAML remove = YAML::Perl remove = YAML::Syck remove = YAML::XS remove = t::lib::Test

With that, all I need to do is run dzil release and -- tada! -- a new CPAN::Meta::YAML is created, tested and released to CPAN. And whenever a new YAML::Tiny is released, I just run dzil release again. It's that easy.

Hooray for Dist::Zilla!

I've written quite a few new modules in the past few months, but I haven't really written about them. Here's a summary of recent(ish) releases.

Silki

(Ok, this isn't really recent, since the first release was in May.)

Silki is a wiki application built with Catalyst and Fey::ORM.

Silki is a multi-tenant wiki hosting system. Translated into English, that means that once you install Silki, you can host many wikis, each of which can have totally different sets of users. Each wiki also has its own set of access controls, from "guests can edit" to "members only".

One of my main goals for Silki is to make it as easy to use as possible. That means a couple things. First, it means not adding tons of features. A lot of wikis seem to suffer from geekitis, with dozens of fascinating and confusing minor features. Second, I'm trying to build an easy to use UI, though whether that's been achieved is highly debatable. It's still missing at least one key feature in this department, a wysiwyg editor.

If you're in need of some wiki software, please take a look. I've worked on making it easier to install than a lot of Perl apps on CPAN, though there's still a long way to go. See the Admin manual for details. However, note that it requires Postgres 8.4+.

Dist::Zilla::Plugin::Conflicts

If you've been around on IRC, you might've heard me whining about the lack of conflict support in the Perl module toolchain. This module integrates Jesse Luehr's Dist::CheckConflicts with Dist::Zilla. This isn't a substitute for support in the installer itself, but it's still useful.

If you maintain a module with a significant number of downstream dependencies, please consider declaring conflicts in your releases.

Antispam::httpBL and Antispam::Toolkit

The Antispam::Toolkit module is a framework for writing spam-checking modules like Antispam::httpBL. The latter uses the Project Honeypot HTTP blacklist to generate a spam score for a user. I have some unreleased work on an Antispam::StopForumSpam module that I need to finish that also uses the Antispam::Toolkit.

My ultimate plan is to write a spam checking tool that can be used to check user-submitted content (or just the users) for spamminess. I want something that does for blogs, wikis, forums, and other web apps what SpamAssassin does for email.

Pg::CLI

This distro provides simple Perl OO wrappers around several of the Postgres command line utilties. For now, it only supports psql, pg_dump, and pg_config. Patches to add support for other utilities are welcome.

Pg::DatabaseManager

While writing Silki (and another webapp I need to blog about) I wrote a lot of database management code. This code deployed a schema, ran migrations, etc. I really wanted to make this code reusable, so I packaged it up as Pg::DatabaseManager (and Pg::CLI).

If you're using Postgres and you want to automate database installs and updates, this may be helpful. It's designed to be subclassed, but I haven't really documented that part of the module yet. If you use this, please let me know how you used it so I know what I need to document.

As a bonus, it also provides what I think is a rather clever tool for doing automated tests of your migrations.

MooseX::Configuration

This is another piece of code inspired by generalizing something I wrote for Silki. This module lets you add metadata to a class's attributes to associate an attribute with a particular key in a configuration file. It only supports INI-style files (for now?).

It also knows how to read and write config files. The writing is a bit "special" because it includes each attribute's documentation metadata (and attribute defaults) as comments in the generated config file. This lets you create a more user-friendly config file. Remember, the config file is part of your application's interface too! Be kind to the sysadmin who only looks at the config file every six months.

Emplacken

Don't use this yet, it's fairly broken ;)

The concept for Emplacken is to let you manage one or more Plack apps from a set of config files, one file per application. It also knows how to generate the PSGI app skeleton for some frameworks, so you don't actually have to have any .psgi files installed.

The first release also attempted to add support for pid files, privilege dropping, and error logging.

This is all useful stuff, but I think I need to break it out a little different. Support for pid files, privilege dropping, and error logging belongs in some sort of process management code. There are lots of options here, including start-stop-daemon (widely used in some Linux distros like Debian), daemontools (for people who've smoked the D.J. Bernstein crack), and Perl solutions like FCGI::Engine.

The latter, despite the name, really isn't FCGI-specified, and can already be used for Plack apps, but it doesn't support error logs or privilege dropping.

My goal is to make Emplacken a tool that reads config scripts, maybe generates some .psgi files, and then calls some supervisor to start/stop the daemons.

Are there Perl or CPAN site Greasemonkey scripts other than those listed under Perl or CPAN at UserScripts? I'd like to put together a website-modding Item for The Effective Perler.

Start submitting your Perl talks for OSCON 2011! Remember my 2010 advice on what not to do and my 2008 advice on what to do.

• Independent Media Sites in Belarus Reportedly Hijacked During Election, SSL Blocked : duplicate (fake) news sites created, possibly to put out fake stories; also interesting that international HTTPS was blocked.
  (tags: election belarus netfreedom via:malaclyps eff filtering censorship)

I wrote previously about the inability of perltidy to handle the method keyword of Method::Signatures::Simple. Now, Steve Hancock has graciously accepted my patches in the latest Perl::Tidy, including prefilter and postfilter options. This allows me to easily add method support:

Perl::Tidy::perltidy( ..., prefilter => sub { $_ = $_[0]; s/^method (.*)/sub $1 \#__METHOD/gm; return $_; }, postfilter => sub { $_ = $_[0]; s/^sub (.*?)\s* \#__METHOD/method $1/gm; return $_; } );

• The prefilter code substitutes method for sub, and adds a comment
  so we’ll be able to find and convert these back afterwards.
• When perltidy operates on the code, it will just see regular subs and treat them accordingly.
• The postfilter code converts the sub back to method wherever the special comment appears.

I put this in my private Perl::Tidy subclass, along with other tweaks, such as telling perltidy to leave my Moose ‘has’ lines alone (I prefer them to always be on one line).

Then I create my own perltidy script which uses this subclass:

#!/usr/local/bin/perl package main; use JS::Perl::Tidy; JS::Perl::Tidy::perltidy();

A little messy, but it works. I’m a devoted perltidy user, with enforced perltidy-on-commit policy on every project, so I’m glad I don’t have to choose between perltidy and my favorite Perl tweaks.

It has been quite a while since I last wrote. I think this is how 50% of blog entries around the world begin.

I've amassed a bit of a backlog over the last few weeks, and blogging was a part of it. This is the rest of the backlog.

Dancer
Dancer is going great! We've successfully wrote all articles for our Advent Calendar. We actually even wrote a few extra. The community continues to blossom, we are seeing increased traffic in the mailing list and more and more fresh monikers, some of which are rather famous. :)
One of the interesting things I've noticed in a community such as the Perl community (which is, in core, a good thing) is that people tend to make use of developments that are done by the people they know. Dancer was written (and is currently still maintained) but people who aren't very famous in the community (maybe except for Franck Cuny - who deserves to be a superstar :) and that means many community members will not flock to Dancer by default. However, we're seeing that block slowly dispersing and it's very pleasing.

The last Rehovot.pm meeting was about Dancer. I gave a talk about it and Gabor gave a talk about trouble he had with Dancer. This wasn't about complaints but of how he understood things and what he had trouble with. It ignited future patches to SiteMap and Dancer. Pretty cool!

The PEG website is written in Dancer (which is a great honor for us) and I've put some time to help make it better. The only thing missing now is META support for non-WRAPPER Template Toolkit rendering. I'm not sure it's possible but in the worst case it can be delegated to using WRAPPER to provide it.

Also, since there was a lot of discussions on the mailing list, I missed some of it. I need to go over it. There were a few patches done following these discussions which I haven't reviewed yet and I need to do that. We're evolving to a more agile streamline of releases which means we release relatively often and fearless these days. We'll probably have a few more releases soon and might even start working on 1.3x (we already have some features for it).

Social get-togethers
In the last Rehovot.pm we decided to start organizing a pure social get-together, without any talks. It's starting with only four geeks (Gabor, Dotan, Ilan and myself) and we'll start meeting every week at a local bar and pub (which is amusing because I don't even drink), to shot the breeze and talk and geek out.

Tel Aviv.pm revival!
I've recently taken up the baton of organizing meetings of Tel Aviv.pm. This used to be a rather big group and slowly dwindled till no one attended anymore. The meetings were canceled (much like Rehovot.pm now - in which the last meeting was probably the last one for a while). I've been approved a DNS (which still isn't up, unfortunately) and static hosting. I got a graphic flyer (thanks to Miss Ferret) and an informational official invitation that I passed around to people. We're trying to get a big amount of people there. If you're in Israel, consider coming!

I need to go over peoples' questions about the meeting and continue to advertise it in companies.

Email backlog
Some of my regular correspondence has been delayed because of all of this. I need to go over a huge amount of emails, and reply to every one. Some of which requires thinking (ouch!) but I think I'll make it.

RSS backlog
I read a lot of RSS feeds. I ordinarily wake up with 100 to 200 unread items to go over. I pretty much read all of them. It starts with going over CPAN uploads, reading changes for everything I use (which isn't a small amount), reading usages of stuff I don't which I might be interested in and checking out stuff I recognize but don't necessarily intend to use.

Since this is the advent season, I have a lot more to read since I subscribed to many calendars.

Chores backlog
Lately I've been living "agile", which probably means nothing to you. Part of this (and you'll get to understand the rest, if I ever publish the post I wrote about it) is to spend every weekend cleaning, tidying and throwing away (usually translates to "giving away" and "donating") stuff I don't need or use frequently. This week I fixed up a quality home surround sound system that I'll be giving to my brother.

Books backlog
I also started reading books again. I'm spending a weekend to a week on each book. Last one was Modern Perl (see: "Modern Perl" below) and the one before that was the Donnie Darko book (which included the entire transcript of the movie). The next one is Flashforward by Robert J. Sawyer. It's a birthday gift I got (along with Damnation Game by my favorite author, Clive Barker, which will be the book after). Modern Perl took me an entire week (a bit at a time) because I was really stressed for time. This week has much more stuff but I hope I'll finish Flashforward by next weekend.

Modern Perl review
I was really really impressed with the Modern Perl book. I've ordered one copy for myself and 4 others to have available at the next Tel Aviv.pm meeting, a week and a half from today. I want to write a blog post dedicated to it (or even a few). The book is better than I even imagined and truly belongs to my "must read" list from now. Any student I'll ever have from this point will have to start with reading this book.

Patches?
Like I wrote above, the PEG development brought an important SiteMap patch idea. To remove regex routes from listing and allow custom listing per route. This isn't difficult to accomplish and I can do that. I just need to find some time.
Also, Tomas Doran is now maintaining MooseX::ConfigFromFile (to which I contributed a patch to allow multiple configuration files) and has asked me to review my patch and add a test file to assure it is working (some versions were released after my patch submissions which didn't include it) and I had already received an email from Zbigniew Lukasiak that included a test for it. These people are part of the reason the Perl community is great!

That's it for now. Gotta start working!