The Nested Procedure Menace in Oberon

Just stumbled across this article and interview with Niklaus Wirth, from July 2 this year. There are quite a few interesting things in there, and also historical facts which are good to know. I’ve run into Wirth a couple of times when I was doing my master thesis at ETH. In fact, I’ve done my thesis in the group which was originally headed by Wirth and now is headed by Gutknecht (also a well-known name from the article).

That’s also when I detected the following flaw in Oberon. Imagine the following module:

MODULE NestedProcTest;

VAR SomeProc: PROCEDURE(val: LONGINT);

    PROCEDURE ProcA();
    VAR someLocalVar: LONGINT;

        PROCEDURE NestedProc(val: LONGINT);
        BEGIN
            someLocalVar := val
        END NestedProc;

    BEGIN
        someLocalVar := 1234H;
        SomeProc := NestedProc;
        NestedProc(5678H);
        SomeProc(0DEADH)
    END ProcA;

END NestedProcTest.

On line 15 we assign the nested procedure (‘NestedProc’) to the procedure variable ‘SomeProc’. This is valid because the signatures match, right? Wrong. The signature for ‘NestedProc’ is different: Because nested procedures have access to the parent procedures’ variables, their signature is added a hidden parameter, the so called Static Link. If this parameter is not passed to the procedure when it’s called, the procedure is going to read either its local variables or its parents’ variables from a wrong place (depending on the actual implementation of the calling convention). This of course is wrong and can even make this code crash in case we’d read from an unallocated or protected region.

That’s why I recommended disallowing the assignment of nested procedures to procedure variables. Back in 2006 I have also proposed a change for the Oberon compiler which comes with BlueBottle, but I cannot tell at this point whether it has made it in to the release or not.

This and other findings can be found in the report for my master thesis (dated March 2006): Improved Stackmanagement in the Active Object Kernel – Master Thesis, Roger Keller (742.85 kb)