BEGIN UPDATE (5/14/08). This solution worked for me for at least 2 months, since then I've had at least 4 reocurrences, all of which were fixed just by reinserting the earphone repeatedly, except for the last instance, I had to clean the earjack and seems back to normal
One day I got a call on my iPhone and I was not able to hear any audio. After a while I figured out I could still hear audio if I put it in speakerphone mode. Next I realized that the play icon was showing up with no audio. I tried pausing/playing music and I could no hear any audio at all. After doing some research on Google I found out this was a very common problem on the iPhone, and unfortunately it appears to be a problem on the hardware and not on the software.
The iPhone has a mechanism to detect when you unplug the headset, apparently this mechanism can fail sometimes due if debris gets in the headset jack. This is why you can't hear any audio, because it thinks you have a headset plugged in and continues to route all audio to the headset output. There are several workarounds published out there, some very creative that suggest using a damped q-tip. But the one solution that worked for me was plugging and unplugging the headset several times until it gets back into the correct state. However this solution was only temporary for me, eventually the iPhone would get into that same state. Because I connect the iPhone on the aux port of my car stereo I had to check every time I left my car to make sure I would be able to use my phone throughout the day!
This became very annoying, so I called Apple Support (1-800-MY-IPHONE) and the workaround they had me try was to plug and unplug my headphones 5 consecutive times as fast as I could. The problem was fixed immediately, but I was very skeptical this would be a long term solution. So I continued verifying that the phone was in the correct state everytime I unplugged the headset. To my surprise this problem hasn't reocurred since then and it's been over 4 weeks and I plug/unplug it at least twice a day, so it seems in my case this problem was indeed caused by a debris and not a permanent hardware problem, which is a relief. So there you have it: the official recommendation from Apple support if you want to give it a shot.