This is a huge challenge for the customers I’ve worked with who have decided to consider an IaaS cloud to host their application. I work with a lot of federal customers and they have a mandate to look at cloud alternative prior to expanding existing data centers. When looking at traditional server focused applications this is a little less of an issue since those workloads are pretty portable across clouds.
By portable I mean there’s not much difference between an instance of Windows or Linux running on Rackspace vs. AWS. I can recover those instances from backup on either infrastructure. The challenge we run into outside of the obvious security concerns are what API’s to commit to when you want to build cloud aware applications.
Don’t get me started on the frustration of trying to offer a brokering service that CloudFoundry is supposed to be for these customers.