My first example is MSG. It causes thirst and headaches in some people. I saw a brand of instant noodles claiming to be MSG free. This does not mean it is replaced by some natural ingredient, instead the noodles contains some chemical I didn't recognise, so I stuck to the tried and tested MSG.
The other example is acetone free nail polish remover. People know acetone is bad for them, so a nail polish remover without acetone must be better, right? Maybe you'd prefer MEK, almost identical to acetone, gamma butyrolactone http://www.ncbi.nlm.nih.gov/pubmed/16035208 or good old highly toxic methanol?
Other examples I could think of:
Very often, the replacement isn't safer, it's just that we don't know how dangerous it is yet. I believe some replacements will be found to be worse than the original in the future.