A vesicle is a small enclosed part of a cell, like a bubble with walls of lipid, that contains something the cell needs to move about. Exocytosis is when the cell moves a vesicle from its interior towards its cell membrane. The vesicle then fuses with the cell membrane (like two bubbles joining and making a slightly bigger one), and the contents of the vesicle are then released.
Most of the time, the vesicle contents are expelled from the cell, but occasionally they are incorporated into the cell membrane.
Cells using exocytosis include neurones (they release neurotransmitters in this way), hormone-producing cells and immune cells (producing antibodies, say). The cell wall and its constituents can also be modified by exocytosis - e.g. the process can be used to incorporate a particular receptor into the cell wall.
Hope that helps!
The above image is from Wikipedia - it rather nicely shows a neurone with vesicles (filled with neurotransmitter). Some of the vesicles are heading towards the plasma membrane, one has just fused with it, and is releasing its contents into the synaptic cleft (the space across which neurotransmitter molecules must diffuse before signalling to the next neurone to fire).