BPA stands for Bisphenol-A, a hormone disruptor found in many baby bottles. It is a chemical used to make hard plastic, namely the recyclable #7, polycarbonate. As a hormone disruptor, it can cause abnormal growth of organs, including the brain.
You should look for baby bottles that say "BPA-free" on their packaging or another alternative is to use glass bottles now available much more readily than they were just a few years ago (BabiesRus - Evenflo, Born-Free, and others)