crypto: ghash - Add PCLMULQDQ accelerated implementation

PCLMULQDQ is used to accelerate the most time-consuming part of GHASH,
carry-less multiplication. More information about PCLMULQDQ can be
found at:

http://software.intel.com/en-us/articles/carry-less-multiplication-and-its-usage-for-computing-the-gcm-mode/

Because PCLMULQDQ changes XMM state, its usage must be enclosed with
kernel_fpu_begin/end, which can be used only in process context, the
acceleration is implemented as crypto_ahash. That is, request in soft
IRQ context will be defered to the cryptd kernel thread.

Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

2 files changed, 15 insertions, 0 deletions

diff --git a/crypto/Kconfig b/crypto/Kconfig
index 26b5dd0cb56..fd6871102b6 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -440,6 +440,14 @@ config CRYPTO_WP512
           See also:
           <http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html>
 
+config CRYPTO_GHASH_CLMUL_NI_INTEL
+        tristate "GHASH digest algorithm (CLMUL-NI accelerated)"
+        select CRYPTO_SHASH
+        select CRYPTO_CRYPTD
+        help
+          GHASH is message digest algorithm for GCM (Galois/Counter Mode).
+          The implementation is accelerated by CLMUL-NI of Intel.
+
 comment "Ciphers"
 
 config CRYPTO_AES
diff --git a/crypto/cryptd.c b/crypto/cryptd.c
index 35335825a4e..f8ae0d94a64 100644
--- a/crypto/cryptd.c
+++ b/crypto/cryptd.c
@@ -711,6 +711,13 @@ struct crypto_shash *cryptd_ahash_child(struct cryptd_ahash *tfm)
 }
 EXPORT_SYMBOL_GPL(cryptd_ahash_child);
 
+struct shash_desc *cryptd_shash_desc(struct ahash_request *req)
+{
+        struct cryptd_hash_request_ctx *rctx = ahash_request_ctx(req);
+        return &rctx->desc;
+}
+EXPORT_SYMBOL_GPL(cryptd_shash_desc);
+
 void cryptd_free_ahash(struct cryptd_ahash *tfm)
 {
         crypto_free_ahash(&tfm->base);