Wiggles

So I decided to change all 4 V-belts on the weekend, and after successfully getting them all installed, I fired her up and awaited the screaming banshee noise, which did not occur, so I guess I set the tension properly.

While the engine was warming up, I noticed smoke coming out from the driver's side valve cover, and then a little from the passenger's side valve cover as well. After turning the engine off, I could see what looked like the charcoal that is produced on well done meat from grilling hamburgers and steaks. I could also see a small amount of wet oil emerging.

My guess is that the valve cover gaskets are rotten and are allowing oil to seep through which is becoming bbq'ed on the outside.

1. Should I go ahead and replace the valve cover gaskets
or
2. After removing valve covers and old gaskets, take care of anything else that needs to be done before installing new gaskets and tightening down the covers?
or
3. I have no idea what I'm talking about