You can squash small infestations of aphids on plants with your fingers or blast with a jet of water. If the infection is limited to a few plants, you can use horticultural soft soap or a very diluted solution of washing up liquid to wash off aphids from leaves and buds. If the infection is major, this can be tedious and can cause leaf damage.
For a more holistic approach, encourage natural predators such as ladybirds, hoverfly and lacewing into your garden by planting nectar-rich flowering plants such as wallflowers, sweet williams and African marigolds. Nettles are also good to plant as they attract the nettle aphid, which is a good source of food for lots of aphid eating insects.
Nettle aphids also appear early in the year, so they attract the predators early on in the season and won’t attack your precious plants as they only feed on nettles. Another solution is to try a biological control such as ladybird or lacewing larvae.