This is a simple solution to the problem with links inside sortables. All that was needed was to add handles. Now you can drag around the items by their handles, but the links inside work fine.
Many thanks to Mickster for pointing this out.