It's not being a wuss to not call them back at all - anxiety is terrible!
Would you make an appointment asap to go and tell your Dr how you are feeling (honestly)? Maybe print out what you've posted here or write a bullet pointed list?