Fix frozen pipes as soon as you find them as ice presents a bigger potential problem than just a temporary loss of water service; if the ice stays in there and continues freezing long enough, it will expand and burst the pipe. It’s important that you quickly find and thaw out the ice clog before that happens.
Determine where the pipe is frozen by following the pipe from the non-working faucet back toward the main water line into the house, testing each faucet along the way. Once you find a faucet that works, and the next faucet down the line doesn't, it means the frozen section is likely between those two faucets.
Press your hand along the exterior of the pipe to confirm that it's very cold. There may or may not be frost formations on the outside of the pipe. (Note: If no faucet anywhere in the house works, then the ice clog is likely in the main feed that comes into the basement, before it gets to the main shutoff valve or any faucet.)
Turn off the main water valve. Open the first valve past the frozen section. (Nothing should come out.)
Run a heat gun, or hair dryer set on the hottest setting, along the pipe, working back from the open faucet. Don't leave the heat in one place, but rather move it back and forth gradually, up and down the whole length of the pipe.
Watch for any trickle of water from the faucet. Once it starts, the flow should gradually increase. Continue until it seems to be flowing freely and starts to diminish again (since all the water in the line will eventually drain).
Turn off the faucet. Turn the main water valve back on. Turn the faucet back on to ensure that it's completely clear and running normally.