Home 
Ask Mr. Make Atomic Rules in GNU Make
Ask Mr. Make Atomic Rules in GNU Make
|
| A fundamental law of GNU Make physics is that each rule builds one and
only one file (called a target). OK, there are exceptions to that rule
(which we'll see later), but nevertheless for any normal GNU Make rule,
such as a: b c @command there's only one file mentioned to the left of the :. And that's the file name that gets put into the $@ automatic variable. And it's expected that command actual updates that file. This article looks at what to do if command updates more than one file, and how to express that so that GNU Make knows that more than one file was updated and behaves correctly. What not to do Imagine a command that makes two files (a and b) from the same prerequisites in a single step. In this article I'll simulate such a command with touch a b, but in reality it could be much more complex than that. Here's what not to do: .PHONY: all all: a b a b: c d touch a b At a first glance that looks correct, it seems to be seeing that a and b are built from c and d by a single command. If you actually run this in Make you'll get the output like: touch a b touch a b The command was run twice. In this case that's harmless, but for a real command that does real work running twice is almost certainly the wrong thing to do. Also, if you use the -j option to run in parallel then you can end up with the command running more than once and simultaneously with itself. This is because GNU Make actually inteprets the Makefile as: .PHONY: all all: a b a: c d touch a b b: c d touch a b So, you end up with two separate rules (one that declares that it builds a and the other than says it builds b) that both build a and b. Using pattern rules GNU Make does have a way to build more than one target in a single rule using a pattern rule. Pattern rules can have an arbitrary number of target patterns and will still be treated as a single rule. For example, %.foo %.bar %.baz: command means that files with the extensions .foo, .bar and .baz (and of course the same prefix that will match against the %) will be built with a single invocation of command. For example, suppose that the Makefile were: .PHONY: all all: a.foo a.bar a.baz %.foo %.bar %.baz: command then command would be invoked just once. In fact it's enough to specify that just one of the targets buildable by the pattern rule is required for the command to run: .PHONY: all all: a.foo %.foo %.bar %.baz: command This can be a very useful technique, for example, here's an actual rule from one of my Makefiles that builds a .lib and its associated .dll in one go: $(OUT)/%.lib $(OUT)/%.dll: $(VERSION_RESOURCE) link /nologo /dll /fixed:no /incremental:no \ /map:'$(call to_dos,$(basename $@).map)' \ /out:'$(call to_dos,$(basename $@).dll)' \ /implib:'$(call to_dos,$(basename $@).lib)' \ $(LOADLIBES) $(LDLIBS) \ /pdb:'$(basename $@).pdb' \ /machine:x86 \ $^ Of course, if the files don't have a common part then using a pattern rule won't work. It doesn't work for the simple example I gave at the beginning. But there is an alternative. Using a sentinel file A work around is to introduce a file that's used to indicate whether any of the targets have been built. That turns multiple files into a single file. Here's the original example rewritten: .PHONY: all all: a b a b: .sentinel @: .sentinel: c d touch a b touch .sentinel The rule to build a and b can only be run once because there's only one target specified (.sentinel). If c or d are newer then .sentinel gets rebuilt and hence a and b. If the Makefile asks for either a or b then they are rebuilt via the .sentinel file. The funny @: command in the a b rule just means that there are commands to build a and b but they do nothing. Of course, it would be nice to make this transparent, and that's where the atomic function comes in. The atomic function sets up the sentinel file automatically based on the names of the targets to be built and creates the necessary rules: sp := sp += sentinel = .sentinel.$(subst $(sp),_,$(subst /,_,$1)) atomic = $(eval $1: $(call sentinel,$1) ; @:)$(call sentinel,$1): $2 ; touch $$@ .PHONY: all all: a b $(call atomic,a b,c d) touch a b All that's been done here is that the previous rule has been replaced by a call to atomic. The first argument is the list of targets that need to be built atomically and the second argument is the list of prerequisites. atomic uses the sentinel function to create a unique sentinel file name (in the case of a b the sentinel file name is .sentinel.a_b) and then sets up the necessary rules. Expanding atomic in this Makefile would be the same as doing: sp := sp += sentinel = .sentinel.$(subst $(sp),_,$(subst /,_,$1)) atomic = $(eval $1: $(call sentinel,$1) ; @:)$(call sentinel,$1): $2 ; touch $$@ .PHONY: all all: a b a b: .sentinel.a_b ; @: .sentinel.a_b: c d ; touch $@ touch a b There's only one flaw with this technique. If you delete a or b you must also delete the related sentinel file otherwise the files won't get rebuilt. That can be worked around by having the Makefile delete the sentinel file if necessary by checking to see if any of the targets being built is missing. Here's the updated code: sp := sp += sentinel = .sentinel.$(subst $(sp),_,$(subst /,_,$1)) atomic = $(eval $1: $(call sentinel,$1) ; @:)$(call sentinel,$1): $2 ; touch $$@ $(foreach t,$1,$(if $(wildcard $t),,$(shell rm -f $(call sentinel,$1)))) .PHONY: all all: a b $(call atomic,a b,c d) touch a b Now atomic runs through the targets and if any are missing (detected by the $(wildcard)) the sentinel file is deleted. John Graham-Cumming is Mr Make and Chief Scientist at Electric Cloud, Inc., a Silicon Valley start-up that speeds up builds by 10-20x using cluster technology.
Set as favorite
Bookmark
Email this
Hits: 9848 Trackback(0)Comments (4)
|
TrackBack URI for this entry
Subscribe to this comment's feed |
... This is why GNUmake and friends need to be retired. The core ideas are fine, but the whole system has become a kludgy hack with too much magic. |
Eric Miller
said:
|
... I missed that part. When I got to the "it would be nice to make this transparent" part of the article, I saw that you were using "eval" and stopped reading, since I'm supporting a group that's frozen at 3.80 with its buggy eval. Sorry about that. |
John Graham-Cumming
said:
|
... Eric, I actually cover that problem in the article. My atomic function creates the appropriate code to remove the sentinel file in the case that you mention automatically. John. |
Eric Miller
said:
|
... The one problem I've run into with this technique is when the real files get removed while the sentinel is still around. For example, the commands: rm a make don't cause anything to be made. In the real world, "a" is more likely to be "a.o", and needed for a link step later, which would fail. One way around this problem is an ugly block of conditionals and a recursive make invocation replacing the "@:" command: a b: .sentinel ifneq (,$(wildcard .sentinel)) ifeq (,$(wildcard a)) rm .sentinel ${MAKE} else ifeq (,$(wildcard b)) rm .sentinel ${MAKE} endif endif endif or in slightly cleaner Gnu make 3.81 form: a b: .sentinel ifeq (,$(wildcard .sentinel)) @: else ifeq (,$(wildcard a)) rm .sentinel ${MAKE} else ifeq "" "$(wildcard b)" rm .sentinel ${MAKE} endif I find this workaround to be disappointingly ugly. Is there a nicer way to handle this problem? |
Write comment
You must be logged in to post a comment. Please register if you do not have an account yet.
