It depends which part is small. If the band fits, but the cup doesn't, change the letter but not the number (e.g. from 34B to 34A (if you need a smaller cup) or from 34B to 34C (if you need a bigger cup).
To change the band size, change both the letter and the number, diagonally (e.g. from 34B to 32C (if you need to go down a band size or make it tighter) or from 34B to 36A (if you need to go up a band size or make it looser).
And to change both the cup and the band size, adjust the number but not the letter (e.g. from 34B to 32B (if it's all too loose, go down) or from 34B to 36B (if both the band and cup are too tight, you need to go up)).
To help you understand view the image below:
You can use this International Bra size calculator to determine your bra size.
Women are making so many mistakes when it comes to their bras, from buying them to maintaining them, that it just takes knowing what you're doing wrong first to actually get it right.
Check out Bobbie Smith of Freshpair's advice on what women are getting wrong so they can make peace with these complex garments once and for all.