The Carnegie Museum of Natural History has a great collection of minerals and gems.