Shamir's Secret: A PayPal Near-Disaster

This account recounts a nightmarish incident at PayPal where a flawed implementation of Shamir Secret Sharing, a cryptographic technique for distributing a secret key among multiple parties, nearly caused a catastrophic system failure. The author, a PayPal engineer, explains the process of Shamir Secret Sharing and how he implemented it to improve security by distributing the master encryption key. However, a seemingly minor incompatibility between the Linux and Solaris operating systems, involving a function that truncated long passphrases, led to the team's inability to recover the key. The crisis was ultimately resolved by discovering and correcting the incompatibility. The story concludes with a humorous postscript regarding a backup copy of the key.