Air sealing the unit in rough opening - With a bit of careful knife work, you can pop off the interior casing of the window / door, and see what's going on in the rough opening.
Carefully cut around the casing, through paint or caulking, to minimize the damage.
If there's a need for air sealing around the frame, apply some low expansion spray foam insulation (for gaps bigger than 1/4").
Apply painters tape around the opening (over the existing drywall or finish) BEFORE you spray foam. The foam will expand out of the opening.
Make sure to clean the gap before you spray.
Caulking around the unit - Sometimes, you can create a good air seal just by applying a bead of caulking.
Use interior silicone caulking around the trim and frame of a window or door, filling in any gaps under 1/4"
Keep rags around for cleaning up excess caulking
Have a container of soapy water close by, to dip your finger in when you 'smooth out' the bead of caulking.
Weather strip moving parts - Older windows and doors may just be in need of new weather stripping.
If you are able to remove the unit from it's frame, you can sometimes slide the old weather strip out and replace it.
Install neoprene, Ethylene Propylene Diene Monomer (EPDM) or vinyl weather strip on the frame itself, being careful not to interrupt the closing action of the unit.