How do you decide what idea to pursure?

I have too many ideas on my Ideas board. It's getting overwhelming. I'm in the process now of finding more, but to what end? I think I need to sit down and decide what should be worked on now instead of trying to discover new ones. Most of mine seem to be a novelty, where the problem doesn't seem to be an urgent one. I am considering adding an urgency flag to each idea. Just a guess at how urgent this problem needs solving.

PG talks about the urgency for ideas:

When a startup launches, there have to be at least some users who really need what they're making — not just people who could see themselves using it one day, but who want it urgently.

Patrick McKenzie also talks about this in a similar context:

Businesspeople generally have room in their head for three things: problem #1, problem #2, and Boring Stuff I Don't Have Time To Deal With Right Now. Don't build until you find people for whom this problem is, at least, problem #2. There are plenty of big hair-on-fire problems which can be addressed in the same amount of time as the not-such-a-problem problem you might think you could address as a first shot.

I think an urgency flag on an is a good metric to go off. From 0-10. 10 being the most urgent. This would prioritize what to work on next. What do you guys use as a process of elimination to find out what to work on next?