...Extra cores don't help – parallel code is really hard to write, and not everything can be parallelised - see Bootnote. So instead, more of the system is moving on-chip - GPUs are just the latest major component to be integrated. RAM will follow in time, as will nonvolatile storage if memristors deliver on their promise. Future computers will be marked "no user-serviceable parts inside", and they'll mean it - you can't upgrade or repair a single-chip device.
What this means is that Koomey's Law has taken over: instead of getting faster, processors are getting smaller, cooler-running, more power-efficient - and cheaper. Only a few niche customers will still want specialist graphics cards or petabytes of spinning rust. The clock's ticking for the beige fan-heater market, and the speed-freaks will have to run deskside servers instead - that's where the big fat hot chips are going.
Inside Steve Ballmer