In reflux, the milk that a baby has drunk comes back up into the oesophagus (sometimes called the food pipe) or the mouth, instead of staying in the stomach. Of course, adults can also have a similar problem after eating their food - this can be caled acid reflux - but babies are especially prone to it.
There is a ring of muscle at either end of the oesophagus, called the sphincter. Usually these muscle rings prevent food from coming back up the oesophagus once it has gone down, and reflux happens wheh they don't work properly. In babies this happens a lot because the sphincters are still not fully developed and also the oesophagus is much shorter, so the food (in this case, milk) has a shorter journey. Also, they are usually lying down which makes reflux happen more easily.
You can learn more about the causes and treatment of reflux here.