Your first experience of a new city will probably be of the tourist district. Transport from outlying airports and train stations is usually set up to take people to hotels and other tourist businesses. Once you're checked in, try to get out of that rut quickly.
Find city listings. When you get to a town, look for a local newspaper or arts magazine to find out what's happening locally. If you don't speak the language, find out if there's a local English-language newspaper. Don't just look at the entertainment listings: Attending a political rally can be just as exciting as going to the theater or a concert.
Get on a bus. Instead of taking a tour bus, grab a local bus schedule and spend the afternoon riding around town at random. Get off and poke around if you see anything interesting. Walk instead of taking a taxi. Rent a bike.
Talk to people (part 2). Even more valuable than talking to people before you leave is talking to people after you arrive. Friendly conversations with like-minded travelers and with residents--especially those not connected with the tourist trade--are bound to turn up recommendations about their own favorite places.
Let new acquaintances know that you're interested in seeing the real, everyday side of the city--not just the stock sights. Ask locals where you can go to meet more locals, or whether there are parts of the city or surrounding countryside that are interesting and relatively tourist-free. Travelers who show local people their genuine interest in exploring a city are often overwhelmed by the generosity and enthusiasm of the response.
Most of all, be flexible. A city you thought would be fascinating might just make you tense; another might beguile you into spending more time than you'd planned. An unforeseen activity may entice you, but conflict with your plan to do something you know you really want to do. Being able to change your mind is one of the freedoms and joys of traveling. So is the realization that there's probably no wrong decision.