The answer is (b): The collision ball apparatus will not work the same if one of the balls has a different mass. What actually happens can be seen in an mpeg video by clicking your mouse on the photograph of the unequal mass collision ball apparatus below.
When you release the ball at the left end, starting the action, what happens is actually a series of collisions between two balls: the first ball hits the second one, transferring all of its energy and momentum, the second ball hits the third one, etc. The end result is that the ball on the right end gets all of the energy and momentum, and it moves away from the others, rising to the height from which the first ball was released.
When one of the balls is a different mass, the chain is broken, resulting in a collision where the final momentum and energy is divided between the incoming ball and the one being struck. The motion then becomes more complex: the momentum and energy is divided between several of the balls, as can be seen.
