I am trying to understand what an objective function really is. Every sequence alignment method should have an objective function. When it comes to the needleman-wunsch algorithm, what is precisely the objective function? I know there is a dynamic-programming matrix that should be filled. At each cell (i,j) in the matrix, the value is the maximum of three things: value in cell (i,j-1) - gap penalty, value in cell (i-1,j) -gap penalty, value in cell (i-1,j-1)+substitution score. Is this what is called the objective function ?

Yes. because it is a DP algorithm, the function seeks to maximize the alignment score at each position by taking the best choice given the context. Smith Waterman does the same but does not mind loss of information at certain stages, leading to a partial match.