Home Ask Mr. Make GNU Make meets file names with spaces in them

GNU Make meets file names with spaces in them

  E-mail
Ask Mr. Make
Written by John Graham-Cumming   
Sunday, 01 April 2007 09:28
GNU Make treats the space character as a list separator; any string containing spaces can be thought of as a list of space-delimited words. This is fundamental to GNU Make and space-separated lists abound.  Unfortunately, that presents a problem if file names contain spaces.  This article looks at how to work around the 'spaces in file names problem'.

A Simple Example
Suppose you are faced with creating a Makefile that needs to deal with two files named 'foo bar' and 'bar baz', with 'foo bar' built from 'bar baz'.  I've include single quotes there to make clear that these are file names that include spaces.

A naive way to write this in a Makefile would be:

foo bar: bar baz
    @echo Making $@ from $<

That doesn't work at all.  GNU Make can't differentiate between the cases where the spaces there are part of, or not part of, the file names.  In fact the Makefile above is exactly the same as:

foo: bar baz
    @echo Making $@ from $<
bar: bar baz
    @echo Making $@ from $<

Placing quotations marks around the file names doesn't work.  If you try to do:

"foo bar": "bar baz"
    @echo Making $@ from $<

then GNU Make thinks you are talking about four files called "foo, bar", "bar and baz".  GNU Make has totally ignored the double-quotes and split the list of file names on spaces.

One way: escape the spaces with \\
GNU Make does have an escaping operator, \\ which can be used to escape sensitive characters (such as a literal # that musn't start a comment, or a literal % in a pattern that shouldn't be the wildcard character). 

In a file name in a rule \\ can be used to escape a space.  The example Makefile can be rewritten as follows:

foo\\ bar: bar\\ baz
    @echo Making $@ from $<

and it will work correctly.  The \\ is removed during the parsing of the Makefile, but the actual target and prerequisite names correctly contain spaces.   This will be reflected in the automatic variables (such as $@). 
When foo bar needs updating the simple Makefile will output:

Making foo bar from bar baz

The same escaping mechanism can also be used inside GNU Make's $(wildcard) function.   To check for the existence of foo bar you can do $(wildcard foo\\ bar) and GNU Make will treat foo bar as a single file name to look for in the file system.

Unfortunately, GNU Make's other functions that deal with space-separated list do not respect the escaping of the space.  So, for example the output of $(sort foo\\ bar) is the list bar foo\\ and not foo\\ bar as you might expect.

The following table shows the GNU Make functions that accept a list as an argument and whether they respect escaping of spaces.

 
Function Respect \\
addprefix No
addsuffix No
basename No
dir No
filter No
filter-out No
firstword No
foreach No
join No
notdir No
patsubst No
sort No
suffix No
word No
wordlist No
words No
wildcard Yes

So, $(wildcard) is the lone hold out.

This leads to a problem if you have to deal with the automatic variables that contain lists of targets.  Consider this slightly more complicated example Makefile:

foo\\ bar: bar\\ baz a\\ b
    @echo Making $@ from $<

Now foo bar has two prerequisites bar baz and a b.  What's the value of $^ (the list of all prerequisites) in this case?  It's bar baz a b: the escaping is gone, and even if it weren't gone the table above shows that it would be pretty much useless anyway.  $^ is, from GNU Make's perspective, a list with four elements.

A quick look at the definitions of the automatic variables tells use which are safe to use in the presence of spaces in file names:

 
Automatic Variable Safe?
$@ Yes
$< Yes
$% Yes
$* Yes
$? No
$^ No
$+ No

And it gets a little worse... even though the first four automatic variables there are safe to use, the modified versions with D and F suffixes (that extract the directory and file name portions of the corresponding automatic variable) are not.   That's because they are defined in terms of the dir and notdir functions.

Consider this example Makefile:

/tmp/foo\\ bar/baz: bar\\ baz a\\ b
    @echo Making $@ from $<

The value of $@ is /tmp/foo bar/baz as expected, but the value of $(@D) is /tmp bar (/tmp/foo bar would be expected) and the value of $(@F) is foo file (instead of file).

Another way: turn spaces into ?
In "Managing Projects with GNU Make, 3rd Edition" by Mecklenburg there's a different suggestion: turn spaces into question marks.  Here's the original simple Makefile transformed that way:

foo?bar: bar?baz
    @echo Making $@ from $<

Since GNU Make does globbing of target and prerequisite names (and respects any spaces found) this will work.  But the results are rather inconsistent.

If foo bar exists when this Makefile runs then the pattern foo?bar will get turned into foo bar and that value will be used for $@.   If that file were missing when the Makefile is parsed the pattern (and hence $@) stay as foo?bar.

(Of course, there's a another problem: that ? could match something other than a space: if there's a file called foombar on the system the Makefile may end up working on the wrong file).

To get around this problem Mecklenburg defines two functions to add and remove spaces automatically.  Here's the updated Makefile using two functions (sq and qs) to add and remove question marks.

sp :=
sp +=
qs = $(subst ?,$(sp),$1)
sq = $(subst $(sp),?,$1)

$(call sq,foo bar): $(call sq,bar baz)
    @echo Making $(call qs,$@) from $(call qs,$<)

Since we can't be sure whether the automatic variables will or will not have question marks in them (since that depends on the state of the file system when the Makefile is parsed), it's still impossible to use the list-based automatic variables or any GNU Make functions that handle lists.

(Aside, the first two lines above end up putting a single space in the macro sp, it's needed for the definition of sq since the first argument to the $(subst) needs to be a space.)

Frankly, I think Mecklenburg's approach causes more trouble than it's worth: it doesn't give consistent results, the values of $@ and other automatic variable have to be wrapped in a call to a function, the ? could end up matching a non-space and all targets and prerequisites need to be wrapped up with a $(call sq).

My advice
So given that GNU Make clearly has a really hard problem with spaces in file names what can be done.  Here's my advice:
 
  • Don't.  If you can rename your files to not include spaces, that's going to be the simplest path.  I realize that for many people this is impossible, those spaces in file names may have been put there by a third-party.
  • Use 8.3 filenames.  If you are on Windows it may be possible to use short, 8.3 file names throughout.   This will allow you to still have spaces on disk, but avoid them in the Makefile.
  • Use the \\ escaping.  If you still need the spaces, then escape than with \\.  This does give consistent results; you'll have to avoid the automatic variables mentioned above.
If you do number 3 and yet you need to manipulate lists of file names that contain spaces, then the best thing to do is transform them by substituting space with some other character and then change them back again.

For example, the s+ and +s functions here change escaped spaces to + signs and back again.  You can then safely manipulate lists of file names using all the GNU Make functions.  Just be sure to remove the +'s before using these names in a rule.

s+ = $(subst \\ ,+,$1)
+s = $(subst +,\\ ,$1)

Here's a fuller example where a list of source files that contain escaped spaces is transformed into a list of object files and then the object files are used to define the prerequisites of the all rule.

SRCS := a\\ b.c c\\ d.c e\\ f.c
SRCS := $(call s+,$(SRCS))

OBJS := $(SRCS:.c=.o)

all: $(call +s,$(OBJS))


 
John Graham-Cumming is Mr Make and Chief Scientist at Electric Cloud, Inc., a Silicon Valley start-up that speeds up builds by 10-20x using cluster technology.


 

Trackback(0)

Comments (5)add comment

feedSubscribe to this comment's feed

Stephen Niedzielski said:

0
 ...
Short name:
It is my understanding that MS-DOS 8.3 "short names" are not always available due to file system configuration. You can read more about this at http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx, specifically:

..Windows may also create a short MS-DOS (8.3) form of the name, called the 8.3 alias, and store it on disk. This can also be disabled for a specified volume.

For this reason, this may not work for some build systems. Also, short names are only available for existent files and directories. As such, I don't believe this to be a working solution for most build systems.

Globbing:
Globbing filenames, substituting spaces for ?, doesn't seem the best solution. I wish to recommend that you consider globbing with [^!-~] instead. Please note this will not work for vpath or VPATH. If you require vpath/VPATH, consider using rule templates instead.
 
August 11, 2008
Votes: +1

Eric H said:

0
 ...
or use a different "make" program, such as bmake. (http://www.crufty.net/help/sjg/bmake.html)
You'll still have a bit of a problem with getting the filenames _into_ make, but once there everything works as expected, provided you pay a little bit of attention and use appropriate variable modifiers.
 
November 13, 2007
Votes: +0

make said:

0
 ...
I hate gnumake. How hard it is to support quoting in notdir.

$(notdir "Program Files oo" "bar")
 
June 29, 2007
Votes: +0

Patrick Egan said:

Patrick Egan
 ...
Looks like the "" in the example was mistaken for an escape character - All fixed now
 
April 04, 2007
Votes: +0

Harri said:

Harri
 ...
Sorry, but it seems that something has been lost when creating the web page. The first example in "One way: ..." looks exactly the same as the naive example at the beginning.

???
 
April 02, 2007
Votes: +0

Write comment

You must be logged in to post a comment. Please register if you do not have an account yet.

busy
Last Updated on Thursday, 26 July 2007 17:44