Government and industry have created strong encryption that can protect data at rest and in transit. Protecting data in use is a trickier task.
Applications expect certain kinds of data—such as Social Security numbers, credit card numbers and other records—to be in a recognizable format so that they can be accessed. It is not necessarily difficult to encrypt these records, but if the application cannot recognize them when it needs them, the app doesn’t work. This means some data has to remain in clear text or be decrypted to be accessed and used, which is a risk.
The solution to this problem is format-preserving encryption (FPE), which can keep data in a recognizable format while making it unreadable. The National Institute of Standards and Technology has published new guidance for FPE that can help protect sensitive information including financial and healthcare data. Special Publication 800-38G, Methods for Format-Preserving Encryption, is the latest in a series of recommendations for using block ciphers such as the government’s Advanced Encryption Standard (AES) or the Triple Data Encryption Standard (3DES).
“FPE is particularly helpful for retrofitting encryption technology to legacy applications where a conventional encryption mode might not be feasible,” NIST researchers wrote in the latest bulletin from the IT Lab.
Credit card, ID and Social Security numbers—personally identifiable information that must be protected—often are used as identifiers linking records in databases. Applications can use them as indexes to retrieve records when the numbers themselves are not necessary to the application. The ability to retrieve data this way without exposing sensitive information is particularly useful in medical research in which data for a particular patient must be located, but it is not necessary to identify the patient.
The problem is that most digital encryption techniques are intended to be used with binary data—strings of 1s and 0s. But this is not the way many apps use the data. A terminal processing a credit card transaction is looking for a sequence of 16 decimals, 0 through 9.
“FPE is designed for data that is not necessarily binary,” the new guidelines say. Given a set of decimal numerals, the encrypted set will have the same format, including length, as the original data. “FPE has emerged as a useful cryptographic tool, whose applications include financial-information security, data sanitization, and the transparent encryption of fields in legacy databases.”
Researchers began working on FPE as far back as 1997, and practical commercial tools began appearing a decade later. But NIST approval is critical for acceptance by federal agencies, which generally are required to use crypto tools that meet the Federal Information Processing Standards.
NIST’s SP 800-38G uses a model based on a Feistel design, named for German-born physicist and cryptographer Horst Feistel. This technique is used in many block ciphers, which is useful because block ciphers produce encrypted text of the same number of characters as the plaintext. But the block cipher algorithms AES and 3DES are intended to encrypt messages of arbitrary lengths and cannot be used directly for FPE. By applying the Feistel technique to selected data sets such as a Social Security number using AES, the result is an encrypted field that retains its original format.
In other words, useful but anonymous.