[% setvar title Provide a standard module to simplify the creation of source filters %]

This file is part of the Perl 6 Archive

Note: these documents may be out of date. Do not use as reference!

To see what is currently happening visit http://www.perl6.org/

TITLE

Provide a standard module to simplify the creation of source filters

VERSION

  Maintainer: Damian Conway <damian@conway.org>
  Date: 20 Sep 2000
  Last Modified: 29 Sep 2000
  Mailing List: perl6-language@perl.org
  Number: 264
  Version: 3
  Status: Frozen
  Frozen since: v2

ABSTRACT

This RFC proposes that the interface to Perl's source filtering facilities be made much easier to use.

DESCRIPTION

Source filtering is an immensely powerful feature of recent versions of Perl. It allows one to extend the language itself (e.g. the Switch module), to simplify the language (e.g. Language::Pythonesque), or to completely recast the language (e.g. Lingua::Romana::Perligata). Effectively, it allows one to use the full power of Perl as its own, recursively applied, macro language.

The Filter::Util::Call module (by Paul Marquess) provides a usable Perl interface to source filtering, but it is not nearly as simple as it could be.

To use the module it is necessary to do the following:

This last requirement is often the stumbling block. Line-by-line source filters are not difficult to set up using Filter::Util::Call, but line-by-line filtering is the exception, rather than the norm. Since a newline is just whitespace throughout much of a Perl program, most useful source filters have to make allowance for components that may span two or more newlines. And that complicates the filtering code enormously.

For example, here is a minimal source code filter in a module named BANG.pm. It simply converts every occurrence of the sequence BANG\s+BANG (which may include newlines) to the sequence die 'BANG' if $BANG in any piece of code following a use BANG; statement (until the next no BANG; statement, if any):

        package BANG;
 
        use Filter::Util::Call ;

        sub import {
            filter_add( sub {
                my $caller = caller;
                my ($status, $no_seen, $data);
                while ($status = filter_read()) {
                        if (/^\s*no\s+$caller\s*;\s*$/) {
                                $no_seen=1;
                                last;
                        }
                        $data .= $_;
                        $_ = "";
                }
                $_ = $data;
                s/BANG\s+BANG/die 'BANG' if \$BANG/g
                        unless $status < 0;
                $_ .= "no $class;\n" if $no_seen;
                return 1;
            })
        }

        sub unimport {
            filter_del();
        }

        1 ;

Given this level of complexity, it's perhaps not surprising that source code filtering is not commonly used.

This RFC proposes that a new standard module -- Filter::Simple -- be provided, to vastly simplify the task of source code filtering, at least in common cases.

The Filter::Simple module

Instead of the above process, it is proposed that the Filter::Simple module would simplify the creation of source code filters to the following steps:

In other words, the previous example, would become:

        package BANG;
 
        use Filter::Simple sub {
            s/BANG\s+BANG/die 'BANG' if \$BANG/g;
        };

        1 ;

Module semantics

This drastic simplication is achieved by having the standard Filter::Simple module export into the package that uses it (e.g. package "BANG" in the above example) two automagically constructed subroutines -- import and unimport -- which take care of all the nasty details.

In addition, the generated import subroutine passes its own argument list to the filtering subroutine, so the BANG.pm filter could easily be made parametric:

        package BANG;

        use Filter::Simple sub {
            my ($die_msg, $var_name) = @_;
            s/BANG\s+BANG/die '$die_msg' if \${$var_name}/g;
        };

        # and in some user code:

        use BANG "BOOM", "BAM;  # "BANG BANG" becomes: die 'BOOM' if $BAM

The specified filtering subroutine is called every time a use BANG is encountered, and passed all the source code following that call, up to either a no BANG; call or the end of the source file (whichever occurs first).

MIGRATION ISSUES

None.

IMPLEMENTATION

A prototype implementation is available from:

        www.csse.monash.edu.au

and should soon appear on the CPAN.

This prototype requires the Filter::Util::Call module, but it is proposed that a standard Filter::Simple module would be self-sufficient.

NOTE

It is certainly not suggested that Filter::Simple should replace Filter::Util::Call. That module provides much more flexible control over source code filtering, and will still be needed in many cases. It is merely proposed that simplified code filtering covering the common cases ought to be incorporated in the core.

REFERENCES

search.cpan.org