The Armory is a military building which can be found in the Buildings Menu under the "Military" tab.
The purpose of this building is to increase the size of the Training Slots in the Barracks. The more Armories you have the more Units you will be able to train simultaneously, in the Training Slots. That means, you have to fill the Training Slots less often and can produce more Units e.g. over night.
As long as you have the proper resources, you will be able to build the first Armory after unlocking it in the Research Tree. It needs a street connection to the Main Hall, it can be sold, rebuilt and upgraded.
When the Orcs come to your city, you get the possibility to breed them in the Armory, after researching the required technology. These Orcs can be bred even after they leave your city to live on themselves. Orcs are required for the activities done in the Orcs Rally Point. They also come in quite handy for delivering goods to far away provinces.
Each new Armory building or each upgrade of an Armory will increase the training Slot size in the Barracks.