Mapping Magnetic Fields in the ISM: Infrared and Sub-mm Polarimetry