The locally most powerful (LMP) tests of the hypothesis $H: \theta=\theta_0$ against one-sided as well as two-sided alternatives are compared with several competitive tests, as the likelihood ratio tests, the Wald-type tests and the Rao score tests, for several distribution shapes and for location, shape and vector parameters. A simulation study confirms the importance of the condition of local unbiasedness of the test, and shows that the LMP test can sometimes dominate the other tests only in a very restricted neighborhood of $H.$ Hence, we cannot recommend a universal application of the LMP tests in practice. The tests with a high Bahadur efficiency, though not exactly LMP, also seem to be good in the local sense.