The GNU Make Course: Episode 3

All right, it is 2.30 here in Lisbon, so I'm going to go ahead and talk about GNU Make. This is the third in a series of little shows I've been doing about GNU Make itself. If you've been following along since the beginning, I've been creating a very simple makefile which has been converting poems into files called count files, and in fact we can just try it out again and just see what it does. So let me just do a quick make clean and then do a make. So what it's doing is it's taking a bunch of files called .poem and for each one it's making a .count and if we just take a quick look at the makefile itself, how that works is for every poem here in the source directory, we make a file in the object directory which contains a number of words and the number of lines in that file. So it's a very simple thing and the idea is that this is pretending to be a compilation step taking some source code file, in this case a poem, and turning it into an object file, in this case just a number. The idea is that this is independent of any particular language. So that's happening and in fact if we take a look in source you can see I've got a bunch of different poems and if I take a look in obj, okay we've got these count files, like that, a little bit of cleanup there. Let's just get rid of those files in there. Let's just do that again. So there's the source files. There's the equivalent object files. If we just look at one of them, how much is one .count? No, pretty many. I can't remember what I'm doing. There you go. It contains the number of lines, the number of words in that particular poem. Okay so that's a very simple makefile and if you take a look over time I've made this more and more complicated. Right now we've got a phony target which is all, which is the first target, so that's what makes everything. It says to make a bunch of object files and each object file is made as I've shown you from the source file and if you look at the top here we've got a list of source files which it discovers by using this function wildcard to look in the source directory and the source directory is up here and it looks through each of these poets. So this is a for loop, the poets here are Owen, Armitage and Bep and if we just take a look in source you can see those are different ones. So Betjeman, there's three poems, there's one by Wilfred Owen, there's one by Simon Armitage and for each of those as source files and object files it creates a list of object files. So what it does is it transforms them into a file with dot count on the end. So there's one way of one way of doing these things and it also changes the source directory at the beginning to obj. So you get these two variables which give you the list of source code and the list of object files and I added right at the bottom here this little thing if you remember which was to print out a variable so we can actually just go and try that again which is make print source files because that's the list of source files and you see it created them. Again there's the equivalent object files and so if you remember from the last one make files are really dumb what they're doing is looking at lists of strings and matching them against files in the operating system and then building things and so this is this is a list to build this object files this is how to build them and it makes them and you'll see that in here I used a bunch of built-in functions so for each which is something that will loop a wildcard which will do globbing on the file system and find a list of files sort which actually does a sort of a string of a list sorry which alpha alphanumeric sort and subst within what subs does is it substitutes one string for another so in that case it's substituting source slash for ob slash so I used a bunch of those functions and those are all things that are built into GNU Make but it's also possible to define your own functions what I want to talk about in this particular episode is defining user-defined functions and also if statements within within GNU Make and how logic is handled so let's start out just by making a very very simple user -defined function and these are defined just like variables are as you've seen for the variables I've used the convention of uppercase and it's quite common in make files to see uppercase used for for variable names for functions it's very common to see lowercase used for the built -in ones and so I tend to define functions using lowercase to distinguish them from uppercase variables so we can go ahead and just make ourselves a little function in here and what I'm going to do is I'm going to take this subst thing and I'm going to turn it into a function so I'm going to call this function change source to obj this can be anything this can be any sort of name and I'm going to say equals and it's going to do what the subs does so it's going to do this subst source obj of something now um what is it doing well arguments so arguments to functions you define yourselves are actually stored in variables called dollar one dollar two dollar three etc so what we've done is defined a function which will substitute source for object in whatever its argument is and we can go ahead and do call on that so how call works how we call a function when we're calling a built-in function we just refer to it like this it's dollar bracket a bit like a variable reference and then a space and that actually causes going to say oh this is not a variable reference this is a function name and we call it now for the built-in one we can't do that you can't do change source to obj because it doesn't know that that thing is actually a function you have to use a built-in function which is call so what call does is it says call this particular function um and then comma and then you're going to give it the uh the the um string or in this case the argument we're going to change so bear in mind that this does changes source slash the obj slash in something i'm just going to make up a string and then we'll see what it does we're going to go source slash hello world dot poem which obviously doesn't exist now this on its own would do that and return the value of the substituted string which would go nowhere at this point so i'm going to actually store it in a variable i'm going to say test so i can go i can go test so it'll store it in test so test will call that particular function i just created and hopefully we'll do the substitution we want so let's go back over here let's see what happens okay well but may pass into the past fine we haven't got a problem with the way it's parsing and now we've got that variable test so let's just go in and print it now using that print function i had before so you can see the test is obj slash hello world dot poem and if i go back it started out as source slash so this has now worked as um as a a function that can be called without um you know having to know the details of how it operates so we can actually go back and do it put it in here now i'm using colon equals here so i better reorder where this is defined otherwise we won't know it won't know how to access it so we must define it there i'm going to go in here and i'm going to say call change source to obj i'll do this so now i've changed them just took i've got my braces right yep good all right so i'm going to go in here and i'm going to say instead of doing the subs directly i'm going to call this function which will then change source slash to obj slash in the source files list and turn it into the list of object files and i don't need test anymore so let's just clean that up and let's just go back and see if make okay so let's do a make clean and did you do it one yes it did exactly the same thing as it did before because it has defined that variable the same way and we can actually verify that let's just have a look at obj files obj files is there all the counts and you can see it's identical to what it was before so that's the most basic thing you can do you can define a function that does anything and it what happens is when the function is called its parameters are placed in these automatic variables dollar one and if i had more you have dollar two dollar three etc um and then the actual string itself in case this whole thing here then gets expanded and gets sub the substance actually called and the thing actually happens so what let's look at adding an extra variable in here let's suppose that we didn't always want to change source to obj but maybe to something else so let's just modify this so we're going to change source source we're going to say change source to something we don't know what it's going to be and we're going to make that something a a parameter so i'm going to do this so now i've said change source to something and uh in here it's going to change source to whatever we put in the second parameter and it's going to substitute it in the first parameter now if i run this mate file as it is i haven't changed the other reference change source to obj which has now disappeared let's just see what it does because it's kind of interesting to experiment and see what make does if i type make nothing happens there's no error because it can't find that function and in fact let me just go back and do a make clean and try and do make and it still doesn't know what to do why is that well when this call has happened here this function doesn't exist and make just silently carries on as if there was no problem at all so that's a little bit scary so let's just go back and just see what happened to that uh the list of obj files now it's empty right because this function doesn't exist okay so let's make it work let's change this to something and we're changing with this list of things and then we're going to add the final parameter what are we going to change it to well we do want it to change to obj right we want it to change so this is going to be this thing is going to get substituted here this thing is going to get substituted here and this should do exactly what it did before so let's go back let's go it works so that's great so we've changed it now to have that one parameter and let's just for completeness do this the last parameter as well change i'm just going to call it change at this point of course you're going to see how silly this is in a minute because this is actually just as complicated as subs but here it is change so we're going to change obj because i've deliberately inverted the parameters here just to show you that isn't actually directly subs but what's going to happen is um you know obj which is going to go into two source is going to go into three and this thing the sort is going to go into one so this should do what you want yeah lo and behold it does the right thing and if we look at obj files there we go it's absolutely correct okay so that's the basics of uh variables a function call and there's also another thing i'm just going to throw in here a little dollar info i'll come back to what it does in a minute it's going to make you see how it's printed out change here and that's what this little piece of code it is a dollar info is a bit like print it allows you to print something to the console there's actually three things you can do there's dollar info which just gives you literally the string um there's dollar warning let's just try that one which gives you the the string plus where it is in the make file so line nine of this particular make file is where that is i'm sure enough we should be there um and then uh what was used on line nine here you see right here um and that's actually an interesting fact if you ever need to debug something and figure out where something is in make if you put in a dollar warning like this um you will get information about where the variable is used so if you notice that that uh output here said make file colon stop doing that so let me make file colon nine this is line nine this is where change was actually used um and actually if i change this to equals which means it's going to be recursively expanded i won't be actually expanded until it's used uh which means it's going to get used there and then i do make you can see now it's on line 12 because line 12 which is right here is where object files got used it then got expanded which called change and then the warning happened and really what's happening here is there's just an expansion gradually of this string calling functions expanding variables and so it happens when it's used and that's kind of useful for debugging and make files because you can find out where something is with dollar warning i said there were three things uh there's also dollar error uh well let's try it let's see what happens okay notice it says star star star before and then the string which is change and stop and so this is actually terminated execution of the make file it won't work the make file and sometimes that's helpful if you want to deal with some sort of error condition uh okay so that's great we've got um something simple there and we can see now what i was saying is dollar zero this variable actually is the name of the function that's being called and sometimes that can be useful because you can actually call functions recursively and it might be nice to know the name of the function so you can call it again so it's possible to actually make recursive functions um in gnu make and you can do things that are really quite complicated uh using that and using the fact that there is actually an if function built in so let me just go back a little bit let's go back to change um of course change here doesn't actually have to be a fixed string right it could be another variable so let's just let's just do that maybe these are all strings there's no magic here it's a string expansion so what's going to happen here is it's going to go here it's going to say call what well it's a variable reference so go get that variable oh it's a function called change okay there's the change function substitute these strings into there um and off it goes and so let's just make sure that we haven't broken anything there you go that looks good looks like it's running just fine all right so there you go that's a sort of simple thing you can do in a makefile with a function and one of the things this is nice for is simple repetitive string manipulations you might need to do if you need to do something more than once to find a function and call it now sometimes you need to make a decision about whether you should do something or not and there is a built-in function in gnu make called if now if it's kind of unusual and it takes three parameters so there's a you know there's the condition and there's the true part and there's the false part we would sort of write it out like that um and what happens is it looks at this condition and if this condition is true and we'll talk a little bit more true means um this part will get expanded and acted upon and if it's false this part will get acted upon and one of the things that's interesting is what true is now within make a non -empty string is considered to be true and an empty string is considered to be false um actually what happens within gnu make is it strips all white space so if there's any white space around here that will get stripped out and then whatever's there if it's non-empty it's true if it's empty it's false um so we can play around with that so let's do this let's go into this change function and let's make sure that we've actually defined something so uh we're going to check that the parameters to this actually are not are non-empty let me just get rid of this changer because it's unnecessarily confusing when we do that let's go in there change that's fine okay so let's go in here we're going to say let's do a simplest version um yeah okay so let's look at this if statement i've created here so i've changed this function now the first thing it does is it says if dollar one and this means if dollar one the first parameter to change is not empty then do this substitution if it is empty false then do this so this will actually generate an error so this might be what something you want to do if you want to check that something's um correctly set and it can be very useful for checking if you know particular things that are input to the make file are actually set so let's just run this it works fine now let's just make a mistake i'm going to go in here i'm going to say this is dollar one and let's mess up source files so let's say we're going to take source files and i'm going to typo this and i'm going to do that i've made a mistake so scrfiles doesn't exist um so what will happen here is that should be an empty string boom we get an error straight away and that error is on line 12 so we go to line 12 which is here what are we dealing with sorry line 12 is here obj files okay where's obj files is here now let's do something let's just change this back to a a value which is going to be evaluated immediately now it's on line nine okay so we can see there's a problem on line nine here something has gone wrong here and if we find this error message here we can say okay well the change function has got a problem and now we can nest this we don't have to just stop here we say if dollar one then we could do if dollar two right we could go go through like this when you say okay well if that guy is missing then we should we should say parameter two is missing right and then where are we there and then if that one failed then error parameter one is missing my brackets right that's the error that's the f so now if this there's a you're gradually going through here checking whether any of those things are are empty and we could we could build this out so let's just make sure this works fine under normal conditions okay let's just do a quick make clean you can see i'm obsessed with make clean so that works great okay so that because parameter one is now back to back to beginning let's typo parameter two up parameter two is missing so obviously it's caught that and just for completeness let's do the whole thing let's add in here dollar three and finally so this would say if this guy is true i am t if this guy is true i am t and then if this one is true i am t then finally do this thing otherwise go through here and see and then we have to add in so let me get this right so parameter two is missing so make sure i've got this end of my error that's the end of that if and then error change parameter one is missing okay now let's just mess up well let's just check that works i haven't messed up anything great that works let's typo this up there we go parameter three is missing so great so that now works and we can perhaps clean this up a little bit let's just go through uh we keep saying change is missing so let's change change to dollar zero so now let's just change it there there there since we know dollar zero contains the name of the function that seems great that still works all right and then perhaps we want to make this even even simpler by going through and checking uh you know each variable in turn using a for loop so i think we can do that let's have a let's have a go let's say the following okay so okay and then let's tidy up it's good let me just go back to you yeah let's see great okay so what did i do here i did a lot of steps in one go there i'm going to show you what i did and i'm using a few little tricks so let's have a look if you notice what change is actually meant to do is right here at the end it's just subs and if you look at this for each it's going to go through each of the three parameters which are just numbered right one two three and then it's going to get the value and notice this little trick i used before see down here i use this trick of the name of the variable is computed it comes from another variable same thing here so dollar v is going to be one and then two and then three and i can do dollar one dollar two dollar three and so i'm getting the three parameters and then i'm saying if that parameter say dollar one is true is a non -empty string then do nothing see this comma comma here there's the the first the second parameter of if is empty so it becomes absolutely nothing there's nothing to do which means actually an empty string is being returned because what will for each will end up being is the concatenation of all of those all of the results of dollar of this thing here but there's an empty string if it was empty if dollar one let's say was empty then you get here and you do the error and it stops immediately and because we know dollar v dollar zero is the name of the function and dollar v contains the number of the variable dollar whatever that's missing then we can say okay it's missing and that's what this little thing does here it says parameter three is missing it's gone through one two three and found them and you know we can do other things here we can say you know um there we say parameter one uh we could say dollar one if we wanted to and we could change things but this will actually uh you know output this will check the check that everything is correct and if that thing succeeds i none of those dollar errors get called and stop the makefile then yay the subs happens and everything works and the main file is is all working here so this is a nice bit of sort of safety checking on a function and you could you could do that um you know for any any kind of function and we can extend this even further we could make another function which is a function that checks the parameters of a function and call that so maybe we'll do that in a in a future video but this kind of shows you the power of if now one of the things that's weird is this concept that an empty string is false and a uh a non-empty string is true one of the things that i sometimes do in makefiles is actually explicitly have true and false i'm trying to debug things so you can do something like this to find two variables one called dollar t which is t which is a non-empty string and i t for truth and f being an empty string and it can be handy sometimes if you're doing things with logic that's more complicated to to do that now um they're actually in newer versions of can you make there are some direct logic functions you can use so this for each year i've done in here could be actually substituted with an and and what it does is it says if all of these things evaluate to be equal to true then you can go off and carry on and you can use it within if um the reason i did the for each here is it's quite nice because it can tell you which parameter is missing and that might help you debug what the problem is so let's just go back here and put this back okay let's do make clean yeah still working great my little makefile is working great um but the equivalent here at the top here if i was used and would be something like if and or one dollar two dollar three and then you know missing parameter or something like that now sorry actually do this if if all those come out to be good and then what you could do now because this is a single statement you could actually put the subs in here so let's just do that okay so it looks something like that so let's just make that into an alternative version of change let's comment that out i think that should that should work yay that worked because this and what it did let's just go through it it said if this thing was if all of these three things were true then do this otherwise give us an error so that's another way we do with an but i kind of prefer this version because i can give me this particular parameter is missing so that's kind of some of the basics of user defined functions very useful for removing repetitive things from a main file because repetition is you know the root of all evil and it's also useful to combine that with logic like if which allows you to make decisions about things and also for each so if you think about it you've got function calls you've got looping you've got logic with if and or you've got the ability to stop the main file with error so there's a tremendous amount you can do here and also what's interesting is all of this is being done before anything happens in the main file there's no files being built at this point this is all during the setup and so it can be useful to check the sanity of the main file make sure the the variables you're using are correct and check parameters do substitutions that you need to do and then you're ready to run the main file so all of this stuff is actually pretty complicated if you look down the bottom the main file itself this bit is quite simple and this bit starts to get a bit dense and so over time you may want to really lay this out and this is a true programming language right this is a list processing language if you think this is a list here with spaces uh between the between the list items and that's the reason why GNU Make has a hard time with file names with spaces in them like you might see quite commonly on Windows because it'll see the space and think it's a new list item so this is fundamentally something that works on lists and actually what I'm going to get to in probably the next episode is some of the very explicit list of list functions there are so just to show you one today it's at five this thing words gives you a count of the number of items in this list so it's actually telling us there are five items in the list and we can start to use things like words and there are some other things which will actually you can actually pull a specific item out so let me just do one of those it's wordless suppose we want to know what the first thing in the list is it's that particular one the Owen poem so you can actually pull out individual items in a list if you need to and this can sometimes be really useful if you start doing things like splitting a file name into directories or extensions and things like you can split it up so in the next episode what I'll do is I'll get into some of the list manipulation stuff which then gives you a lot of power to do things in a makefile and then in the subsequent episode I'm going to go back to the makefile here and I'm going to finish it off because right now we've got it if you remember kind of taking source code and turning it into object code but we don't have a linking step and so we're going to put in the linking step and we're going to make it look like this this gets linked to some executable all right I think I've got very little time left hopefully this will give you some sense for the power of user-defined functions for if within gnu make and you know good luck with your makefiles thanks very much you