This show has been flagged as Clean by the host.
This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems.
The
echo
command is very useful—it prints the arguments given to it, followed by a newline character. (The newline is sometimes also called a linefeed character depending on who is writing or speaking, and has the ASCII decimal value 10.) It has many uses, either in a script or interactively on the command line. The
echo
utility is used to display text, the value of a variable, or the result of a pathname expansion. It can also feed text to another command in a pipeline.
As useful as
echo
is, it should come as no surprise that it
first appeared early on in Bell Laboratories' Second Edition UNIX
1
in 1972. This
initial version accepted no options
2
—although the manual page doesn't explicitly say output is followed by a newline character, the description of writing "as a line" seems to imply it. In
Seventh Edition UNIX, the manual page
3
makes that clear, and also features the addition of the
-n
option, which causes
echo
to print the arguments
without
a trailing newline character.
Eighth Edition UNIX's
echo
4
gained the
-e
option, which allows certain escape codes from the C programming language to be used.
These variations caused differences in behavior between different versions of
echo
. Will running
echo -n something
on your system output the text "something" without a newline, or "-n something" followed by a newline? Things get even trickier when the command arguments include parameter or pathname expansions. If there are files named "-n" and "something" in the current directory, what does
echo *
output? Like the previous question, that depends on whether or not your version of
echo
treats
-n
as an option. You can't get around this ambiguity by quoting or escaping the "*", because that just causes
echo
to print a literal asterisk.
Example using GNU utilities on Debian 12; both the "echo" utility and the "echo" builtin of bash recognize "-n" as an option.
$ ls -1
-n
something
$ echo *
something$ #Shell prompt is on the same line because "-n" was treated as an option to echo
$ echo "*"
*
The solution was to create a new utility, which is the first UNIX Curio for today:
printf
. This command allows a user to print text similar to the way the identically-named function works in the C programming language. You
run
printf
5
followed by a format string, followed by zero or more arguments. No newline characters are printed unless specifically indicated by the format string or the arguments.
To use
printf
to print "something" without a newline, that would just be
printf something
. This demonstrates that you don't need any arguments—in this example, the format string is just a set of regular characters to be displayed. If you wanted a newline character at the end,
printf "something\n"
would give you that. (In this case, the format string needs to be quoted so the "\n" isn't interpreted by the shell.) In addition to "\n" for a newline, you can also use "\a" for an alert (rings the terminal bell), "\b" for a backspace, "\f" for a formfeed, "\r" for a carriage return, "\t" for a horizontal tab, "\v" for a vertical tab, and "\\" to get a literal backslash. In addition to these special characters, any arbitrary byte can be included using a backslash followed by one to three octal digits; however, it might be difficult to predict what will be output because it can differ based on the character set the terminal is using. It is safer and more portable to stick to the pre-defined characters if possible.
The
real
magic of the
printf
utility comes from using "conversion specifications" in the format string. Probably the simplest of these to explain is the "%s" conversion specification—it represents a string of any length. The command
printf "Hi, %s, how are you?\n"
followed by a list of names would print the greeting for each name, putting it in the place occupied by the "%s".
$ printf "Hi, %s, how are you?\n" Alice Bob Carol
Hi, Alice, how are you?
Hi, Bob, how are you?
Hi, Carol, how are you?
The format string is reused as many times as needed to consume all of the arguments. Take, for example, the command
printf "Hi, %s, have you met %s?\n"
. If this is run with two name arguments, it would print the sentence on one line, using both names. If run with four name arguments, it would print the sentence twice, once with the first two names and again with the second two names. If you only gave it three names, the last "%s" conversion specification would be replaced with a null string.
$ printf "Hi, %s, have you met %s?\n" Alice Bob
Hi, Alice, have you met Bob?
$ printf "Hi, %s, have you met %s?\n" Alice Bob Carol David
Hi, Alice, have you met Bob?
Hi, Carol, have you met David?
$ printf "Hi, %s, have you met %s?\n" Alice Bob Carol
Hi, Alice, have you met Bob?
Hi, Carol, have you met ?
Three other items can also be given in each conversion specification: flags, the field width, and the precision. The exact meanings of these depend on which type of conversion specifier character you are using. For "%s", using a "-" as the flag causes the text to be left-justified instead of the default right-justified, a field width causes the printed field to be at least as long as the number given, and a precision limits the number of bytes written from the string to the number given.
$ #Example of %s with a precision value
$ printf "Hi, %.3s, how are you?\n" Alice Bob Carol
Hi, Ali, how are you?
Hi, Bob, how are you?
Hi, Car, how are you?
$ #Example of %s with a field width
$ printf "Hi, %8s, how are you?\n" Alice Bob Carol
Hi, Alice, how are you?
Hi, Bob, how are you?
Hi, Carol, how are you?
$ #Example of %s with a left-justify flag and a field width
$ printf "Hi, %-8s, how are you?\n" Alice Bob Carol
Hi, Alice , how are you?
Hi, Bob , how are you?
Hi, Carol , how are you?
$ #Example of %s with a left-justify flag, a field width, and a precision
$ printf "Hi, %-8.3s, how are you?\n" Alice Bob Carol
Hi, Ali , how are you?
Hi, Bob , how are you?
Hi, Car , how are you?
While "%s" is probably the most commonly-used conversion specification, others are available. A whole set of them are dedicated to printing integer values as a signed decimal, an unsigned decimal, an unsigned octal, or an unsigned hexadecimal number. These also can take flags, a field width, and a precision. I think the details and nuances of all this are too complex to clearly explain here, so I will just refer you to the
POSIX "file format notation" specification
6
.
Be aware that unlike the
printf
function in the C programming language, the
printf
utility is
not
obligated to accept conversion specifications for floating-point numbers. While some implementations might support this, scripts intended to be portable should limit themselves to the restricted set required by the POSIX standard (%d, %i, %o, %u, %x, %X, %c, and %s, plus %b and %% described below).
Two more conversion specifications are worth mentioning. The first is
only
required by the standard for the
printf
utility, not the C function, and is "%b". This is the same as "%s", except that certain backslash escape sequences in the argument will be treated specially. This includes all the ones described above
except
for the one using octal digits to represent a byte. In an argument, this is instead represented by "\0" followed by one to three octal digits. An additional backslash escape sequence accepted is "\c"—this does not print anything itself, but causes
printf
to immediately halt output.
The final conversion specification is "%%", which just outputs a literal "%". You can't use a bare "%" in the format string, because
printf
expects that to introduce a conversion specification. Be careful not to be tripped up by this when trying to print some value as a percentage.
Example assuming that the hypothetical "/dev/batterycharge" file on your laptop outputs the battery charge level (42% in this case). As you can see, in some cases an error message might be displayed, but in others it might just behave in a way you didn't intend without complaining. GNU's "printf" utility and the "printf" builtin of bash both support "%e" as a conversion specification as an extension to POSIX.
$ cat /dev/batterycharge
42
$ #Wrong
$ printf "Your laptop's charge level is $(cat /dev/batterycharge)%.\n"
bash: printf: `\': invalid format character
Your laptop's charge level is 42$ #Shell prompt appears here from the error
$ #Right
$ printf "Your laptop's charge level is $(cat /dev/batterycharge)%%.\n"
Your laptop's charge level is 42%.
$ #Next one treats %e as the specifier, with the space and "l" as flags
$ printf "Your laptop has $(cat /dev/batterycharge)% level of charge.\n"
Your laptop has 42 0.000000e+00vel of charge.
$ #Because no arguments were given, "0" was used for the value to convert
Let's go back to the situation I was describing with
echo
—we have files named "-n" and "something" in the current directory and want to print all their names, separated by spaces. We could do that with
printf "%s " *
, which would not treat the "-n" as an option. However, the output might look a little weird because there wouldn't be a newline character at the end. We could insert a newline by using "%b" instead of "%s" and following the asterisk with a "\n\c" as the second argument. The "\c" is there to prevent the final space in the format string from being printed after the newline.
$ ls -1
-n
something
$ printf "%s " *
-n something $ #No newline was printed here
$ printf "%b " * "\n"
-n something
$ #There's a newline, but also a spurious space before the shell prompt
$ printf "%b " * "\n\c"
-n something
$ #No space before the shell prompt this time
Using the "%b" conversion specification can therefore solve one problem, but it also introduces another. Arguments which include a backslash can be interpreted as escape sequences, and many systems are fine with allowing backslashes in filenames. In cases where you're just using the
printf
utility to
display
text, it's usually not a big deal if the output looks a little wonky. Where you really need to be careful is when the text is being piped to another program, as control characters and other oddities might cause unexpected results, and can potentially create security problems if processed by a script or utility running as a privileged user.
$ #GNU "ls" displays filenames containing a backslash in single quotes
$ ls -1
apple
banana
'\cherry'
durian
$ printf "%b " * "\n\c"
apple banana $ #"\c" in "\cherry" stops output immediately
The
printf
utility
looks to have shown up first in 1986's Ninth Edition UNIX
7
, though the
earliest manual page I could find
8
is from the Tenth Edition. Its first appearance in BSD
seems to be from 1990 in the 4.3 Reno release
9
. Two years later, it was added to Issue 4 of The Open Group's CAE Specification. From what I can tell, it did not seem to be in AT&T's System III—presumably the
printf
utility did make it into System V at some point but I found it difficult to track this down.
While
echo
is still suitable for use where you know for certain that you want a newline character printed at the end and none of the arguments will start with a hyphen, consider using the
printf
utility instead for displaying text. It offers more flexibility and features than you are guaranteed to get with
echo
, although it does require a bit of forethought in constructing a proper format string and arguments. That is not necessarily a bad thing, because a script's author
should
be thinking about what might happen if it is called with "strange" text or filenames.
This episode also provides a good case for being careful when naming files—many filesystems will allow you to use hyphens, control characters, quotation marks, and potentially any character other than a slash or a null byte in a filename. As we've seen, some of these characters can create problems for standard utilities. While it can feel limiting, especially for people not using English, the safest filenames to use on a UNIX-like system consist only of characters in the
"portable filename character set" as defined by POSIX
10
and where the first character is
not
a hyphen. This set includes the lowercase and uppercase letters "a" through "z", the numerals "0" through "9", and the period, underscore, and hyphen. Notably, it does
not
include the space character.
That leads me to another UNIX Curio that I only just now discovered while researching this episode. This is
the
pathchk
utility
11
. It can be run with one or more strings as arguments, checks each one against a set of rules for pathnames, and outputs an error message for each problem found. By default, it checks against the following limits on the system where it's being run: maximum number of bytes in the full path, maximum number of bytes in any component of the path, all byte sequences must be valid in the given directory, and the user running the program must have access to all directories referenced. If run with the
-p
option, instead of those limits, it checks against POSIX limits: a maximum of 256 bytes in the full path, a maximum of 14 bytes in each component of the path, and each component must only include characters from the portable set. The
-P
option adds warnings if any component starts with a "-" or if the pathname is completely empty. While the exit status will tell you if the checks succeeded or not, I don't feel like the
pathchk
utility is well suited to be used in an automated fashion, as the exact wording of its output is not specified and checks cannot be selected individually. However, it can be used interactively to validate pathnames you aren't sure about. See the linked specification for full details.
References:
A Research UNIX Reader: Combined Tables of Contents
https://archive.org/details/a_research_unix_reader/page/n99/mode/1up
A Research UNIX Reader: Second Edition UNIX echo manual page
(although this page has "v1" typed at the top, the date and the tables of contents indicate it first appeared in v2, a.k.a. Second Edition) https://archive.org/details/a_research_unix_reader/page/n22/mode/1up
Seventh Edition UNIX echo manual page
https://man.cat-v.org/unix_7th/1/echo
Eighth Edition UNIX echo manual page
https://man.cat-v.org/unix_8th/1/echo
Printf specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html
File Format Notation specification
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap05.html
A Research UNIX Reader: Ninth Edition Table of Contents
https://archive.org/details/a_research_unix_reader/page/n95/mode/1up
Tenth Edition UNIX echo/printf manual page
https://man.cat-v.org/unix_10th/1/echo
4.3BSD Reno printf manual page
https://man.freebsd.org/cgi/man.cgi?query=printf&sektion=1&manpath=4.3BSD+Reno
Definitions: Portable Filename Character Set
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282
Pathchk specification
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pathchk.html
Provide feedback on this episode.
Netherlands