In this paper, we consider the Swift-Hohenberg equation with perturbed boundary conditions. We do not a priori know the eigenfunctions for the linearized problem since the ${\rm SO(2)}$ symmetry of the problem is broken by perturbation. We show that how the neutral stability curves change and, as a result, how the bifurcation diagrams change by the perturbation of the boundary conditions.