DDoS attacks are still a serious security issue on the Internet. We explore a distributed Cloud setting in which users are mapped to servers where malicious users mapped to the same server can thwart the performance of legitimate users. By periodically shuffling the mapping of users to servers and observing how this affects successfully attacked servers, the malicious users can be identified. We use simple models to understand how to best score these observations to identify malicious users with well-defined levels of confidence.