Click on the Layer Miniature icon, and select Filter → Blur → Surface Blur
from the menu. Play around with the Radius and Threshold sliders until the
noise has been reduced as much as possible. Thanks to the mask you
created, the contours are safe.
With and without noise