Replace format built-in with format function
Maintainer: Damian Conway <[email protected]> Date: 15 Sep 2000 Last Modified: 20 Sep 2000 Mailing List: [email protected] Number: 230 Version: 3 Status: Frozen
This RFC proposes that Perl's existing format mechanism be replaced
with a standard module based on parts of the Text::Autoformat module.
I can never remember how formats work. The specification syntax is confusing to me. And I usually don't want the formatted text going straight down some magical output stream.
It all came to a head when I was building Text::Autoformat. The smart text
recognition was easy -- trying to do the final formatting with formline
and $^A was just too painful.
So I created the Text::Autoformat::form subroutine. It uses a template
specification than fits my brain better, it's deeply DWIMical, it's highly
configurable, and it's re-entrant (so you can use a form to format
another form's headers and footers, for example).
I propose that the existing format mechanism be removed from Perl 6
and be replaced with an add-in function from a standard module (based on
the semantics of Text::Autoformat::form) as described in
the following sections.
Format::format functionThe function itself would be called format and would be
imported with a use Format call.
It takes a format (or "picture" or "template") string followed by one or more replacement values. It then interpolates those values into each picture string, and either:
returns the result as a single multi-line string (in a scalar context)
returns the result as a list of single-line strings (in a list context)
In a void context, format would emit a warning: "Useless use of format in void context".
A picture string consists of sequences of the following characters:
Left-justified field indicator. A series of two or more sequential <'s specify a left-justified field to be filled by a subsequent value.
Right-justified field indicator. A series of two or more sequential >'s specify a right-justified field to be filled by a subsequent value.
Centre-justified field indicator. A series of two or more sequential ^'s specify a centred field to be filled by a subsequent value.
A numerically formatted field with the specified number of digits to either side of the decimal place. See Numerical formatting below.
Left-justified block field indicator. Just like a <<<<<<<<< field, except it repeats as required on subsequent lines. See below.
Right-justified block field indicator. Just like a >>>>>>>> field, except it repeats as required on subsequent lines. See below.
Centre-justified block field indicator. Just like a ^^^^^^^^ field, except it repeats as required on subsequent lines. See below.
A numerically formatted block field with the specified number of digits to either side of the decimal place. Just like a >>>.<<<< field, except it repeats as required on subsequent lines. See below.
A single character field. Interpolates a single character from a subsequent data value. Repeats on subsequent lines as required.
A zero-width field separator. A null character in the template field is ignored (i.e. not included in the formatted text). This allows two fields to be adjacent. For example: "[[[[[[[[[[[\0[[[[[[[[[[[". To put a literal "\0" in a formatted text, use a '~' field and pass it the data string "\0".
Literal escape of next character (e.g. \~ is formatted as a literal '~',
not a one character wide field).
That literal character.
Hence a typical use of format might look like this:
$formatted = format "<[[[[[[[[[[[[[[[[> >>>>>>>>>>>>>>>>",
$aphorism, "page $page_num";
and might produce an output like this:
<Like a camel > page 123
<through the eye >
<of a needle, so >
<are the days of >
<our lives >
Note that, because every field (except a ~ field) must be at least
two characters wide, the single < and > brackets in the
format string are correctly interpreted as literals.
As the previous example indicates, any line with a block field continues interpolation of that field on subsequent lines until all block fields in the format have consumed all their data. Non-block fields on these lines are replaced by the appropriate number of spaces.
For example:
$title = "On The Evil That Is Spam";
$text1 = "How many times have you longed to smash...";
$text2 = "...the bedevilment that is spam?";
print format "<<<<<<<<<<< [[[[[[[[[[[[[[[ [[[[[[[[[[",
$title, $text1, $text2;
# prints:
#
# On The Evil How many times ...the be-
# have you longed devilment
# to smash... that is
# spam
As the previous example indicates, within a block field, words are wrapped whole, unless they will not fit into the field at all, in which case they are broken and hyphenated. Simple hyphenation is used (i.e. break at the N-1th character and insert a '-'), unless a suitable alternative subroutine is specified instead.
Words will not be broken if the break would leave less than 2 characters of the word on the current line. This minimum can be varied by setting the 'minbreak' option to some positive numeric value. This value indicates the minumum total broken characters (including hyphens) that must be left on the current line.
Alternative word breaking subroutines can be specified using the "break" option in a configuration hash. For example:
print format { break => \&my_line_breaker },
$template,
@data;
format expects any user-defined line-breaking subroutine to take three
arguments (the string to be broken, the maximum permissible length of
the initial section, and the total width of the field being filled).
The subroutine must return a list of two strings: the initial
(broken) section of the word, and the remainder of the string
respectively).
For example:
sub tilde_break = sub($$$)
{
(substr($_[0],0,$_[1]-1).'~', substr($_[0],$_[1]-1));
}
print format { break => \&tilde_break },
$template,
@data;
This causes '~' to be used as the hyphenation character.
The standard Format.pm module provides two exportable functions to simplify the use of variant hyphenation schemes.
The exportable function Format::break_with takes a single string
argument and returns a reference to a sub which hyphenates with that
string. Hence the previous example could be rewritten:
use Format qw( break_wrap );
print format { break => break_with('~') },
$template,
@data;
The exportable function Format::break_TeX takes an optional argument
specifying a TeX hyphenation file and returns a reference to a sub which
hyphenates using Jan Pazdziora's TeX::Hyphen module (assuming it is
installed). For example:
use Format qw( break_TeX );
print format { break => break_TeX("lithuanian.hy") }
$template,
@data;
Note that in the previous examples there is no leading '\&' before
break_with or break_TeX, since each is being directly called,
arther than referred to. They each return a reference to some other
suitable word-breaking subroutine.
Specifying {break = undef}> reverts format to its default line-breaking
behaviour (see A note about format options and Lexically permanent options below).
The Format.pm module might also provide other predefined breaking
subroutines; for example, to cope with localization and two-byte
character issues such as those addressed by existing modifications to
the Perl 5 format mechanism in JPerl.
format preserves the original whitespace (including newlines) from an
interpolated string, unless called with certain options.
The "squeeze" option (when specified with a true value) causes any interpolated sequence of spaces and/or tabs (but not newlines) to be replaced with a single space.
The "fill" option independently causes newlines to be "squeezed".
Hence:
$frmt = "# [[[[[[[[[[[[[[[[[[[[[";
$data = "h e\t \tl lo\nworld\t\t\t\t\t";
print format $frmt, $data;
# h e l lo
# world
print format {squeeze=>1}, $frmt, $data;
# h e l lo
# world
print format {fill=>1}, $frmt, $data;
# h e l lo world
print format {squeeze=>1, fill=>1}, $frmt, $data;
# h e l lo world
Whether or not filling or squeezing is in effect, format can also be
directed to trim any extra whitespace from the end of each line it
formats, using the "trim" option. If this option is specified with a
true value, every line returned by format will automatically have the
substitution s/[ \t]+$//gm applied to it.
Hence:
print length format "[[[[[[[[[[", "short";
# 11
print length format {trim=>1}, "[[[[[[[[[[", "short";
# 6
format optionsLike templates and data, option sets (passed via hash references) can
be interleaved within a single format call.
For example, to leave the text in $quote unsqueezed, unfilled, and unhyphenated, whilst squeezing, filling, and hyphenating the text in $intro and $commentary:
print format
{ squeeze=>1, fill=>1, break=>Format::break_TeX },
'[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]',
$intro,
{ squeeze=>0, fill=>0, break=>undef },
' > [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[',
$quote,
{ squeeze=>1, fill=>1, break=>Format::break_TeX },
'[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]',
$commentary;
or, more cleanly:
$TIDY = { squeeze=>1, fill=>1, break=>Format::break_TeX };
$RAW = { squeeze=>0, fill=>0, break=>undef };
print format
$TIDY, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]',
$intro,
$RAW, ' > [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[',
$quote;
$TIDY, '[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]',
$commentary;
Note that -- in Text::Autoformat::form -- changes in options cannot
currently be made between interpolated data arguments (i.e. they
apply to the next complete interpolation template and all the data
interpolated into it).
However, the ability to apply options to a single interpolated
field might be a useful feature in the proposed Format::format. For
example, to create two columns where the left column is (only) filled and
the right is (only) squeezed:
print format
'[[[[[[[[[[[[[[[[[[[ [[[[[[[[[[[[[[[[[[[',
{ squeeze=>0, fill=>1 }, $column1_text,
{ squeeze=>1, fill=>0 }, $column2_text;
format consumes stringsWithin a single call to format fields consume data
text as they format it, so the following:
$text = "a line of text to be formatted over three lines";
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
produces:
a line of
text to
be fo-
not:
a line of
a line
a line
To achieve the latter effect, the variable arguments must be converted to independent literals (by double-quoted interpolation):
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
"$text", "$text", "$text";
Although values passed from variable arguments are progressively consumed
within format, the values of the original variables passed to format
are not altered. In other words, internally format operates on a
copy of each variable's string. Hence:
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
$text, $text, $text;
print $text, "\n";
will print:
a line of
text to
be fo-
a line of text to be formatted over three lines
To cause format to visibly consume the values of the original
variables passed to it, they must be passed as references. Thus:
print format "<<<<<<<<<<\n <<<<<<<<\n <<<<<<\n",
\$text, \$text, \$text;
print $text, "\n";
will print:
a line of
text to
be fo-
rmatted over three lines
Note that, for safety, the "non-consuming" behaviour takes precedence,
so if a variable is passed to format both by reference and by value,
its final value will be unchanged.
The ">>>.<<<" and "]]].[[[" field specifiers may be used to format numeric values about a fixed decimal place marker. For example:
print format '(>>>>>.<<)', <<EONUMS;
1
1.0
1.001
1.009
123.456
1234567
one two
EONUMS
would print:
( 1.0 )
( 1.0 )
( 1.00)
( 1.01)
( 123.46)
(#####.##)
(?????.??)
(?????.??)
Fractions are rounded (a la sprintf) to the specified number of
places after the decimal, but only the minimal number of significant
digits are shown. That's why, in the above example, 1 and 1.0 are
formatted as "1.0", whilst 1.001 is formatted as "1.00".
It is possible to specify that the maximal number of decimal places always be used, by giving the configuration option 'numeric' the value 'AllPlaces'. For example:
print format { numeric => 'AllPlaces' },
'(>>>>>.<<)', <<'EONUMS';
1
1.0
EONUMS
would print:
( 1.00)
( 1.00)
Note that although decimal digits are rounded to fit the specified width, the integral part of a number is never modified. If there are not enough places before the decimal place to represent the number, the entire number is replaced with hashes.
If a non-numeric sequence is passed as data for a numeric field, it is formatted as a series of question marks. This querulous behaviour can be changed by giving the configuration option 'numeric' the value 'SkipNaN', in which case, any invalid numeric data is simply ignored. For example:
print format { numeric => 'SkipNaN' },
'(>>>>>.<<)',
<<EONUMS;
1
two three
4
EONUMS
would print:
( 1.0 )
( 4.0 )
Other values for the "numeric" option might also be provided, such as:
Locale-sensitive commification of numbers. For example
print format {numeric=>'Commas'},
']]]]]]]]].[',
[100,1000,10000,1000000,10000000];
# 100.0
# 1,000.0
# 10,000.0
# 1,000,000.0
# #########.#
Cause the locale's currency sign to be properly inserted as part of each interpolation. This option would also imply 'AllPlaces'. For example:
print format {numeric=>'Currency'},
']]]]].[[',
[1,100,1000,10000];
# $1.00
# $100.00
# $1000.00
# #####.##
Use the specified character as the fill character, rather than spaces. For example:
print format {numeric=>'Fill(*)'},
']]]]].[[[[',
[1,10.01,100.001,1000.0001,10000.00001],
{numeric=>'Fill(+)'},
']]]]].[[[[',
[10,100,10000];
# ****1.0***
# ***10.01**
# **100.001*
# *1000.0001
# 10000.0000
# +++10.0+++
# ++100.0+++
# 10000.0+++
If the data provided for a particular field is an array reference,
then format automatically joins the elements of the array into a single
string, separating each element with a newline character. As a result, a
call like this:
@values = qw( 1 10 100 1000 );
print format "(]]]].[[)", \@values;
will print out
( 1.00)
( 10.00)
( 100.00)
(1000.00)
as might be expected.
Note that, because arrays must be passed using a reference, their
original contents are consumed by format, just like the contents of
scalars passed by reference.
To avoid having an array consumed by format, pass it as an anonymous
array:
print format "(]]]].[[)", [@values];
The format subroutine can also insert headers, footers, and page-feeds
as it formats. These features are controlled by the "pagenum", "pagelen",
"header", "footer", and "pagefeed" options.
The "pagenum" option takes a scalar value or a reference to a scalar variable and starts page numbering at that value. If a reference to a scalar variable is specified, the value of that variable is updated as the formatting proceeds, so that the next page number is available in the variable after formatting. This can be useful for multi-part reports.
The "pagelen" option specifies the total number of lines in a page (including headers and footers, but not page-feeds).
If the "header" option is specified with a string value, that string is used as the header of every page generated. If it is specified as a reference to a subroutine, that subroutine is called at the start of every page and its return value used as the header string. When called, the subroutine is passed the current page number.
Likewise, if the "footer" option is specified with a string value, that string is used as the footer of every page generated. If it is specified as a reference to a subroutine, that subroutine is called at the start of every page and its return value used as the footer string. When called, the footer subroutine is passed the current page number.
Both the header and footer options can -- alternatively -- be specified as hash references. In this case the hash entries for the keys "left", "centre" (or "center"), and "right" specify what is to appear on the left, centre, and right of the header/footer. The entry for the key "width" specifies how wide the footer is to be. The "left", "centre", and "right" values may be literal strings or subroutines (just as a normal header/footer specification may be.) See the second example below.
The "pagefeed" option acts in exactly the same way as "header" and "footer", using a literal string or a subroutine to produce a pagefeed string (but not necessarily "\014"), which is appended after the footer. Note however that the pagefeed is not counted as part of the page length, nor can it be specified as a left-centre-right hash.
The header, footer, and pagefeed page components are recomputed at the start of each new page, before the page contents are formatted (recomputing the header and footer makes it possible to determine how many lines of data to format so as to adhere to the specified page length).
When the call to format is complete and the data has been fully formatted,
the footer subroutine is called one last time, with an extra argument of 1.
The string returned by this final call is used as the final footer.
So for example, a series of 60-line pages with appropriate headers and footers might be formatted like so:
print format
{ header => sub { "Page $_[0]\n\n" },
footer => sub { return "" if $_[1];
"-"x50."\n".format ">"x50", "...".($_[0]+1);
},
pagefeed => "\n\n",
pagelen => 60
},
$template,
@data;
Note the recursive use of format within the "footer" option.
As a second example, to set up headers and footers such that the running head is right justified in the header and the page number (starting at 7) is centred in the footer:
print format
{ header => { right => "Running head" },
footer => { centre => sub { "Page $_[0]" } },
pagelen => 60,
pagenum => 7,
},
$template,
@data;
If use Format is called with a hash reference as an argument, the
entries of that hash specify options that are to be made the defaults
until the end of the current lexical scope. For example, to cause
format to always squeeze and trim whitespace but respect newlines,
the following line would be placed at the start of the source file:
use Format { squeeze=>1, fill=>0, trim=>1 };
With these defaults in effect, filling could be turned on and trimming disabled within a particular subroutine like so:
sub particular {
use Format { fill => 1, trim => 0 };
# do filled, untrimmed formatting here
} # format's defaults revert to previous values at end of scope
format examplesAs an example of the use of format, the following:
$count = 1;
$text = "A big long piece of text to be formatted exquisitely";
print format q{
|||| <<<<<<<<<<
----------------
^^^^ ]]]]]]]]]]\|
=
]]].[[[
}, $count, $text, $count+11, $text, "123 123.4\n123.456789";
produces the following output:
1 A big long
----------------
12 piece of|
text to be|
formatted|
exquisite-|
ly|
=
123.0
123.4
123.456
Alternatively, picture strings and replacement values can be interleaved. For example:
$report = format
'Name Rank Serial Number',
'==== ==== =============',
'<<<<<<<<<<<<< ^^^^ <<<<<<<<<<<<<',
$name, $rank, $serial_number,
''
'Age Sex Description',
'=== === =====================',
'^^^ ^^^ [[[[[[[[[[[]]]]]]]]]]',
$age, $sex, $description;
Non-trivial, but do-able.
I also intend to extract Text::Autoformat::form from Text::Autoformat
and create a new Perl 5 module: Text::Reformat.pm. This would provide
the same functionality (and a migration path) for Perl 5 users.
See Text::Autoformat module.
The proposed built-in/add-in ought to be reimplemented in C for speed, with Text::Autoformat::form (or Text::Reformat::format) as a reference.
None.