Straight from the tree, olives are hard and bitter. To remove the bitterness, you have to cure them. After they have been cured they can be stored with all kinds of flavorings, like for instance lemon, oregano, thym and garlic.
Check out Milkwood for some curing methods.