Well, I think a good indicator would be to go up to your local mountains (are there any near you?) for the day. Eeek has the right idea about altitude. The south entrance, on the 41, doesn't climb much either. The valley, itself, really isn't that high. Now if you go up the 120 or camp off it (White Wolf, Toulmne Meadows), be prepared for elevations (over 8,000 or 9,000 ft., in some places) and cold nights.
I think your biggest concern is keeping the baby warm, at night. Bring rated (a 30 degree rating should be fine) sleeping bags. Yosemite Valley is still pretty warm (80s usually) during the day.
All in all, I think the baby will be fine as long as you come over prepared, like us mothers tend to do. lol Just make sure to bring all of your infant-type medications because I'm not too sure how vast the selections are at the stores. I know they sell over-the-counter meds, for adults. Same goes for fomula, baby food and diapers. I do know that everything in the stores, there, run a lot more, than if you were to bring it yourself.
The valley also has a medical center, so if the baby does have a bad earache, keep that in mind.