Which of two comfort stations are we talking about here? There's the one near the Vernal Fall footbridge, with flush toilets and a drinking fountain/water station nearby; and the one just above Vernal Fall, with composting toilets and no water supply that I know of. From the mention of the drinking fountain, I'd guess this is the one at the footbridge, not the one at Vernal Fall.
I'm pretty sure the Footbridge water station has its own local water supply, separate from the Valley. In July, 2012 I topped up my water bottle there before tasting the water. I discovered, too late, that it reeked of chlorine. A year later I was at the same spot and made sure to filter the water before putting it in my bottle. When I tasted it afterwards, straight from the faucet, it turned out to taste fine.

When I'm headed that way I make sure to fill up at Happy Isles. The water there has been consistently excellent. I'm guessing it's from the same supply as the Visitor Center, campgrounds, etc.