Yann Kerhervé's blog

recent perl mistakes (or quite mistakes)

What to write about today? I don't have much time for iron man, so I decided to share two recent bummers or quite.

 Yesterday I got hit by the following code written by someone else:

if (! $blog->can('allow_html_comment') or ! $blog->allow_html_comment) {
   remove_html();
}

First, it's probably better to cleanly define the class/role hierarchy to avoid can() altogether, but sometimes, using can() is just the cheap hack that make it Just Work, so let's let it go.

What I was bitten by is actually in the underlaying allow_html_comment() implementation which I co-wrote. &allow_html_comment is actually installed using AUTOLOAD(), so can() always return false until the first call inside the process, making this conditional evaluation inconsistent depending on the execution order. Aweful.

I guess can() is a semi-bad idea, AUTOLOAD used this way is another semi-bad idea, but both combined make it a really-bad idea.

The second bummer is in this gist: http://gist.github.com/107815

It never occurred to me to use scalar() the way it is used in 'baz', but it occurred to a colleague, do you know what it does and why?

more review-board and git

I tried to read more of review-board documentation, and it looks like the post-review tool I was talking about in an earlier post doesn't do exactly what I want... Besides, it doesn't work when there is a relative path on the subversion server. I've dirty patched it (no guarantee at all that it doesn't break something else) to make it do something instead of failing (see gist below).

On the other hand, it looks like it still insists on using 'master' one way or another which doesn't really work for me. Ideally I would have to modify that too, or maybe add an option to the tool to make it accept a svn diff in input, because surprisingly it doesn't exist (post-review really wants to do the diff itself, which is... noble).

Posting a patch.diff based review right from the command line instead of using my browser would still be a total win, anyway.

 

post review and git

I found a nicer solution to my problem of making review-board work with Git. Indeed I've read the manual! so intead of my svn-diff.pl: you can also install the command-line post-review tool that also does tons of other useful things. With that tool do:

post-review --parent trunk -n

to generate a diff that you can upload in review board. I'm searching for the right way of actually submitting this patch directly to a svn review-board repository (review board can apparently be hooked into a git repo directly too), but so far I haven't read in the manual that it was possible.

Find::Lib to unclutter tools, tests and other stuff

In the past I've been typing over and over again the following kind of things:

use FindBin qw($Bin);
use lib "$Bin/../../../app/lib";
use lib "$Bin/../locallib/";
...

or,

use FindBin qw($Bin);
use lib "$Bin/../../../common/lib";
use Bootstrap; # smartely imports additional libraries for my app

Especially in test files, I use this all over the place. The nice concept with the exported $Bin variable is that it allows you to specify local libs path relative to the given script or tool, making the code transportable easily.

But now, what I do, is I install everywhere I can Find::Lib module, that provides some nice benefits:

• reduces the typing to one line in most cases
• no crufty $Bin thing which doesn't even indicate a binary
• since version 0.04 handles well the case where symlinks are used all over the place (which is not the case whith FindBin)

The second example is advantageously rewritten to:

use Find::Lib '../../../common/lib" => Bootstrap;

In conclusion, wherever you can install Find::Lib in system PERL5LIB, I recommend you use Find::Lib. Everywhere else you can use the buggy FindBin which is in Perl core, it is still better than using absolute paths in the source. Another solution is to use $ENV variables to indicate an application "base", but personnally I dislike it because I feel like I'll forget to update the env variable.

Find::Lib on CPAN

update: I just found out rlib on CPAN that has been around for 9 years. Find::Lib is more capable though

Perl hash autovivification is hateful

Perl Iron Man rules specifically said we could blog about what we hate; the nasty habit that Perl has to autovivificate hashes when you don't expect it to, is one of them.

autovivification in itself is a good thing and is useful in many places [1], but at times, it's just plain aweful:

Here is a first nasty effect in a common idiom:

yann@dev:~$ re.pl                                                                 
$ my %hash =  ( x => 1);
$ exists $hash{x}                                                                   
1
$ exists $hash{meta}
$ exists $hash{meta}{z}
$ exists $hash{meta}
1

And now here is a second common idiom that can be fatal:

$ my $hash = {}
$HASH1 = {};
$ for (keys %{ $hash->{meta} }) {                                                   
> print "$_\n";                                                                     
> }                                                                                 
$ $hash
$HASH1 = { meta => {} };

Each time, the 'meta' hash has been autovivicated when deferenced, the problem is that it's quite easy to oversee that fact and somewhere else do in the code:

unless (exists $hash->{meta}) {
    do_something_really_important($hash); # will install $hash->{meta}
}
## code that relies on $hash->{meta} to have meaningful informations

Where you can, completely avoid to evaluate conditions based on the presence of a hash in your hash, because this can lead to silent and painful bugs. Calling method calls on objects where it's possible is one option, or use another indicator that do_something_really_important() needs to be called...

To finish with a bit of love after all this hate: re.pl is awesome and you can get it here (checkout MultiLine PPI plugin too):

http://search.cpan.org/perldoc?Devel::REPL

[1] http://www.sysarch.com/Perl/autoviv.txt

SPAM: maladresse ou malveillance?

Je me suis inscrit il y a plusieurs années sur le site LAMBDA, pour voir, et je n'ai jamais consenti a ce que mon adresse email soit partagée. Et ce matin, je reçois un SPAM d'un site qui n'a rien a voir si ce n'est qu'il est hébergé par le même contact technique.

Que ce soit de la négligence ou un acte purement criminel de la part de Mr. FC, le gérant de ce site: c'est tout simplement lamentable.

update:

En plus des commentaires ci-dessous avec FC et Renaud Varoqueaux., des mails ont ete echanges, et j'attends d'y voir un peu plus clair.

update 2:

J'ai censure ce post, ne voulant pas porter trop de prejudices a quelqu'un qui n'avait pas d'intentions malveillantes (mais qui a porté prejudice a de nombreux internautes). FC a fauté professionellement et est responsable de cette erreur, en admettant qu'il s'agisse bien d'une erreur, meme si ce n'est qu'un prestataire qui l'a commise.

Perl script to help stuffing diff into Review Board

update: I found another way

At Six Apart we occasionally use Review Board to submit patches for review. For a while I've been using Git with git-svn to work on our proprietary projects. So my workflow looks like:

$ git checkout -b feature-X
... work ...
$ git commit # multiple times
... work ...

# go back to my branch where I track subversion's trunk
$ git checkout svntrunk

# get changes from upstream
$ git svn rebase

$ git checkout feature-X
$ git rebase svntrunk

So then I need to send the patch for review, but a simple git diff svntrunk doesn't work with review board. I guess there is a better way (I even found git mentioned in Review Board's UI) but I couldn't find it. So I started with a simple shell script to take git diff output and change it to be accepted by review board, but in the last version Perl came to the rescue to make it a bit better and easier to read (So this entry counts for Perl Iron Man).

Here it is:

This removes a/ and b/ prefixes that Review Board rejects, add the subversion revision number and remove /dev/null that git uses to mark new files.

this thread gave me the inspiration: http://www.nabble.com/Using-git-svn-with-review-board--td16119879.html

Perl Iron Man

my Feed reader come up with this post:

Blog once per week [about Perl].

Every week.

Every single week.

Ten lines is fine. It's going to take all we failures some effort to get into this, so there's no going to be restriction on post length. There's going to be no barrier for how badly written it is. None of us are good enough, and that isn't a problem. What matters is we try. What matters is that we make some fucking noise. Show off cool new toys you've found, or cool new toys you're writing, or even trying to write. Show off neat tricks you came up with. Talk about your failures, too, because then maybe the next guy who makes the same stupid mistake will find it on google and think "I'm so glad that guy didn't mind sharing his idiocy because he's saved me wasting half a day because I'm an idiot too." Tell the world that perl5 is alive. If you're a perl6 true believer tell the world how it's soon going to be christmas. Talk about the stuff that you love, the stuff that you hate, the stuff that excites you, the stuff that drives you insane. But MAKE SOME FUCKING NOISE because if enough of us do, the law of averages says there's going to be some of it that's well worth reading.

So, I've set up an alarm in my calendar and I'll try to do just that... Blogging and Perl: I had to do it... won't be pretty most likely, but not posting is even worse.

What about you?

I like moose

The past weeks I've been a bit more interested in Moose and I liked the concepts so far. At the surface It's still the same old Perl but with less typing and incentives to reuse best practices, but when you dig a bit further the MOP is really kicking ass.

In order to get accustomed with it I've started to port Data::ObjectDriver to it (you can find it in my github repo). I just have the new 02-moose.t test passing. Another reason to look into that (Yet Another ORM) is that there are some parts in DOD that I really like and that aren't in Fey::ORM or Reaction AFAICT.

The whole process gave me ideas about something else, so I have dropped this branch and I have spent some of my free time on this new idea. I'll post here if this gets interesting.

Git-svn hack to import original committers from an svk import of svn

I don't recall exactly what was the context but on one of your repo at Six Apart we had the beginning of the project imported into the public repostitory using an external tool (svk I think), I made a patch to git-svn (quick and dirty) to look into the subversion log message and correctly attach the commit to the original author.

The original commit messages have this format:

        # r123@host (orig r1234):  author | 2006-05-17 17:07:42 -0700

subversion to git in a corporate environment

Lately Git is really the latest "Cool" SCM, a lot of its appeal is coming from GitHub. I like a lot the features that GitHub is providing on top of git. The other day I was hearing a colleague mentioning: "Isn't that ironic for a decentralized scm to take off once you start centralizing projects on GitHub?".
While this is certainly amusing, this is not exactly true, github is nothing like a centralized place, I see it has a rally point (especially for opensource stuff), a convenient hub from where you can share stuff, but thanks to git, it's in no way an obligatory gateway to the code. You can clone / mirror all the stuff from github in a snap, painlessly.

Back to subversion... This is what we use at Six Apart, internally but also externally (for the numerous opensource repository we maintain), and on daily basis I find it painful especially when you compare to git.

Part of the pain comes from merging and branching (some of the pain apparently could go away with the latest subversion I heard), and this pain is amplified by the fact that we have tons of svn:externals So MAKING a release at Six Apart, is not easy (Would you believe me If I told you that tectonic was involved in that process? (no, not tectonik, at least not yet))... I'm still unclear how we would replace, or better, work with a subversion repository with that many externals.

So these difficulties set aside, I was wondering how other big software shops are dealing with such transition or how they have changed (improved) their release management using git?

Another pain right now is that our dedicated QA team is dependent on engineers to do merges and don't really own the release. With git I was musing that ideally QA people would be actually signing off a release by actually branching and tagging in their "Golden QA repo". I kind of like the idea that QA would pull from engineering repository, merging all of the different feature/bug branches that they want to see in a release.

Conceptually, their work should consist in, pull from one bug-branch from one engineer, build, verify. If it's not good communicate the problems to the engineer, and if it's good merge in the "signed-off repo".

In realily I don't know if its doable. Anyone knows a big software shop working with Git, what are the obstacles, best practices?

GetSatisfaction with Whole Foods Market

Thanks, it took sometime but I finally got some answer today from a Whole Foods rep:

The price tags are unclear and don't make price comparison easy between products.

According to the folks at our Northern California regional office, they have recently switched their shelf-tag system. The region is exploring options to get the unit price back on the tags and they are hopeful that a software update will do the trick.

If only ZipCar could reply the same way. I was about to open a thread on GetSatisfaction, but I lost faith in ZipCar, I don't think they are listening. I'm considering bringing my money to the non-profit citycarshare instead.

Terminal.app vs iTerm

I'm giving another shot at iTerm:

pros:

• 256 colors, yeah
• quite fast
• less weird screen wrapping (to be confirmed)
• I guess I could fix and hack on it if I really wanted

cons:

• no way to have confirmations before closing mortal sessions (vim), and NO confirmation for resistant ones (screen)
• what's the shortcut to create a new *window* out of this bookmark/profile whatever?
• 0 or 10% transparency ? I want 5%!
• The whole window profile thing is a lot easier on Apple's Terminal

Improved bash prompt for git usage

I've found some interesting gists on github. There are many flavor of a bash prompt showing which branch you're on, and if you have uncommited changes.

I've forked mine here it is:

http://gist.github.com/52483





I'm not sure I'll keep it though, since I'm using it on my virtualbox (with shared folders) and it makes the prompt rather slow. Anyway it's kind of cool, I also have it natively on my mac and I don't have any speed issue.

libpurple fixes

I've finally found the time last time to look into the problem Adium had to connect to DJabberd using DIGEST-MD5 of SASL.

I wasted some time at first because, by default the trunk in subversion builds libpurple with cyrus-sasl which is immune to that problem, so I had to disable it to test the patches. Here are the patches:

http://pidgin.im/pipermail/devel/2009-January/007402.html
http://pidgin.im/pipermail/devel/2009-January/007409.html

Adium + cyrus-sasl can do auth-conf. Cool! I should test that (not really useful, as people should just do STARTTLS+PLAIN)