21.6. Newline Convention

21.6. Newline Convention
Prev	Chapter 21. Streams chap-21	Next

21.6.1. Should programs output a newline before or after each line of output?
21.6.2. Analysis
21.6.3. Conclusion
21.6.4. Solution
21.6.5. Elastic Newline Analysis

21.6.1. Should programs output a newline before or after each line of output?

The answer is complicated. There is an antagonism between the “old Lisp way” of outputting a newline before the line's contents (exemplified by the functions PRINT and PPRINT) and the “Unix way” of outputting a newline after the line's contents. Which one is “right”?

A newline convention is, by definition, a consistent way to use the TERPRI and FRESH-LINE functions or - in FORMAT notation - ~% and ~& directives in such a way that the resulting output is properly subdivided into lines.

Three newline conventions are conceivable:

Print a newline before the line, and nothing after it. As a format string: “~%First line.~%Second line.”
Print a newline if needed before the line, and a newline always after it. As a format string: “~&First line.~%Second line.~%”
Print nothing before the line, and a newline always after it. As a format string: “First line.~%Second line.~%”

The most important criterion is interoperability. Two newline conventions are interoperable if, when parts of a program use one of the convention and other parts of the program use the other conventions, lines are still properly separated. It is easily seen that A and B are interoperable, B and C are interoperable as well, but A and C are not interoperable: When an output with convention A is followed by output in convention C, two lines are appended without a line separator. This should not happen.

Therefore, in what follows, we consider five kinds of programs:

A: using convention A exclusively,
AB: mixing conventions A and B,
B: using convention B exclusively,
BC: mixing conventions B and C,
C: using convention C exclusively.

Which of these five kinds of programs operation is satisfactory? Let us consider different criteria:

Do extra blank lines occur during normal operation?
What happens if FRESH-LINE prints a newline when it is not needed, i.e. when it cannot tell for sure whether the current column is 0? (This situation happens, for example, when logging to a file: After the user has entered a line interactively, the column on screen is 0, but since the input has not been echoed in the log file, the column in the log file is usually not 0, and FRESH-LINE must output a newline. Then a blank line is visible on the screen.)
What happens if FRESH-LINE omits a newline when it would be needed? (This is more rare, but can happen, for example, when standard output and standard error are different streams but are joined outside the Lisp implementation, at the OS level. Such as in lisp | cat.)
Is it possible to reliably output a blank line before or after a paragraph of text? I.e. what happens with
1. 1. “~%~%First line.~%Second line.”
  2. “~%First line.~%Second line.~%”
2. 1. “~&~%First line.~%Second line.~%”
  2. “~&First line.~%Second line.~%~%”
3. 1. “~%First line.~%Second line.~%”
  2. “First line.~%Second line.~%~%”
Is it possible to optimize away blank lines? I.e. is it possible to avoid a blank line even though another piece of code uses one of A1 ... C2, without risking that adjacent lines be unseparated?

21.6.2. Analysis

- A: No extra blank lines.
- AB: An extra blank line each time one switches from convention B to A.
- B: No extra blank lines.
- BC: No extra blank lines.
- C: No extra blank lines.
- A: No extra blank lines.
- AB: Blank lines can occur when convention B is used.
- B: Blank lines can occur.
- BC: Blank lines can occur when convention B is used.
- C: No extra blank lines.
- A: No problem.
- AB: Lines can be unseparated when one switches from convention A to B.
- B: No problem.
- BC: No problem.
- C: No problem.
- A: No problem.
- AB: The blank line is omitted when using A2 before switching to B.
- B: No problem.
- BC: No problem.
- C: No problem.
- A: Yes, using “~&First line.~%Second line.” eats a previous blank line.
- AB: Not really: Using “~&First line.~%Second line.” may eat a previous blank line or a following blank line, but you cannot know in advance which one.
- B: Yes, using “~&First line.~%Second line.” eats a following blank line.
- BC: Impossible.
- C: Impossible. To optimize blank lines in case C would require the opposite of FRESH-LINE, namely a conditional newline that is annullated if the next output on the stream will be a newline. (EXT:ELASTIC-NEWLINE, see below.)

21.6.3. Conclusion

Each approach has its advantages and disadvantages.

When used globally (i.e. no interoperability requirements), A, B, C can be compared as follows:

A and C are equally perfect if eating blank lines is not a requirement.
If eating blank lines is desirable, A is perfect.
B is not so good, because it is suboptimal in case 2.

For CLISP built-ins, however, the interoperability requirement with both A and C is a major requirement. Therefore we have to choose B, and accept the drawbacks:

AB: An extra blank line each time one switches from convention B to A.
B: When logging to a file, blank lines can occur.
AB: When joining two output streams into one, lines can be unseparated.
AB: Blank lines after a paragraph can be eaten by CLISP.
AB: Optimizing blank lines is not really possible.

And to minimize the drawbacks, we recommend the user programs to use approach B or C, but not A.

Another drawback of B is, however, that in interactive sessions the cursor is nearly always positioned at the beginning of a line, pointing the user's focus to the wrong point and taking away a screen line.

21.6.4. Solution

To solve this, we introduce the concept of an elastic newline, output by the function EXT:ELASTIC-NEWLINE. This is the converse of FRESH-LINE: It waits for the next character and outputs a newline when the next character is not a newline; then the next character is processed normally. As a FORMAT directive, we write it ~.. EXT:ELASTIC-NEWLINE followed by FRESH-LINE leads to exactly one newline always.

Elastic newline leads to a slightly different newline convention:

B': Print a newline if needed before the line, and a newline if needed after it. As a format string: “~&First line.~%Second line.~.”

The five programs being considered are now:

A: using convention A exclusively,
AB': mixing conventions A and B',
B': using convention B' exclusively,
B'C: mixing conventions B' and C,
C: using convention C exclusively,

21.6.5. Elastic Newline Analysis

- A: No extra blank lines.
- AB': No extra blank lines.
- B': No extra blank lines.
- B'C: No extra blank lines.
- C: No extra blank lines.
- A: No extra blank lines.
- AB': Blank lines can occur when convention B' is used.
- B': Blank lines can occur.
- B'C: Blank lines can occur when convention B' is used.
- C: No extra blank lines.
- A: No problem.
- AB': Lines can be unseparated when one switches from convention A to B'.
- B': Lines can be unseparated when one switches from one stream to another without performing a FORCE-OUTPUT. This is a general problem with buffered streams; CLISP's FRESH-LINE contains a workaround that is limited to *STANDARD-OUTPUT* and *ERROR-OUTPUT*.
- B'C: No problem.
- C: No problem.
- A: No problem.
- AB': The blank line is omitted when using A2 before switching to B' or when using B2 before switching to A.
- B': No problem.
- B'C: No problem.
- C: No problem.
- A: Yes, using “~&First line.~%Second line.” eats a previous blank line.
- AB': Not really: Using “~&First line.~%Second line.” may eat a previous blank line or a following blank line, but you cannot know in advance which one.
- B': Yes, using “~&First line.~%Second line.” eats a following blank line.
- B'C: Impossible.
- C: Yes, using “First line.~%Second line.~.” eats a following blank line.

Now criterium 1 is satisfied perfectly. We therefore choose B', not B, for use inside CLISP, and programs can use either A or C without problems during normal operation.

Prev	Up	Next
21.5. Non-Blocking Input and Output	Home	21.7. Function `STREAM-EXTERNAL-FORMAT`