Consider what you intend the bike to be used for. Mountain bikes have become very style-specific and buying the right type of bike is crucial. The major categories of mountain bikes are cross-country (lightweight and nimble, designed for all trail use), freeride/downhill (heavy, sturdy, designed for downhill riding, speed and jumps), all mountain (compromise between cross-country and freeride/downhill) and dirt jump (similar to BMX).
Additionally, consider the terrain you'll be riding and decide if you need suspension. If you plan on riding very technical terrain with steep descents and lots of rocks, you will want a full-suspension, but if you'll be riding a mix of pavement and flat, smooth paths, you may not need any suspension at all.
Unlike many other types of sports equipment, mountain bikes are a purchase that should never be done over the Internet. You should always test ride bikes and get a feel for how comfortable they are. Local bike shops are the best source for mountain bikes.
You want to get the most bang and comfort for your buck, so familiarize yourself with different components, including drivetrain components, brakes and shocks. Also familiarize yourself with different features that may be important to you. For instance, some rear shocks and forks offer a lock-out feature that allows you to stiffen the shock so that it does not bob when you are pedaling uphill, allowing for smoother, easier climbing. Other features to consider are disc brakes versus V-brakes, individual geometry and design, the comfort of different types of shifters and details like number of water bottle holders.
When you've found a bike that you like, ask staff to help you determine the frame size that you'll need and then take it out for a test ride. Any bike shop will let you take the bike for a ride around the block or parking lot and this is the most important aspect of buying a mountain bike. While riding, be very critical and get a feel for how comfortable the bike is. Are your arms spread comfortably on the bike or are you scrunched or spread too far? Are you hunched over in an uncomfortable position or are you sitting comfortably? Individual bike geometry can greatly affect your comfort level. Also try out the shifters and brakes to make sure they are easy to operate and work smoothly for you. If you have any issues, discuss with the store staff who can point you toward a more appropriate bike.
It's best to test ride a few different models for comparison. Once you've found a comfortable favorite, you are ready to purchase. Consider any component changes you might like to make (different pedals or saddle, for example) at the time of purchase, because shops often allow you to do this for little or no charge (and likely only any price difference). Also bear in mind that there is a level of markup on any retail product and you might be able to negotiate a cheaper price. Don't be afraid to make an offer.