Multithreaded Keystore
This patch transitions keystore a threading model with one dispatcher
thread and one worker thread per keymaster instance, i.e. fallback, TEE,
Strongbox (if available). Singleton objects, such as the user state
database, the enforcement policy, and grant database have been moved to
KeyStore and were made concurrency safe.
Other noteworthy changes in this patch:
* Cached key characteristics. The key characteristics file used to hold
a limited set of parameters used generate or import the key. This
patch introduces a new blob type that holds full characteristics as
returned by generate, import, or getKeyCharacteristics, with the
original parameters mixed into the software enforced list. When
keystore encounters a lagacy characteristics file it will grab the
characteristics from keymaster, merge them with the cached parameters,
and update the cache file to the new format. If keystore encounters
the new cache no call to keymaster will be made for retrieving the
key characteristics.
* Changed semantic of list. The list call takes a prefix used for
filtering key entries. By the old semantic, list would return a list
of aliases stripped of the given prefix. By the new semantic list
always returns a filtered list of full alias string. Callers may
strip prefixes if they are so inclined.
* Entertain per keymaster instance operation maps. With the introduction
of Strongbox keystore had to deal with multiple keymaster instances.
But until now it would entertain a single operations map. Keystore
also enforces the invariant that no more than 15 operation slots are
used so there is always a free slot available for vold. With a single
operation map, this means no more than 15 slots can ever be used
although with TEE and Strongbox there are a total of 32 slots. With
strongbox implementation that have significantly fewer slots we see
another effect of the single operation map. If a slot needs to be
freed on Stronbox but the oldest operations are on TEE, the latter
will be unnecessarily pruned before a Strongbox slot is freed up.
With this patch each keymaster instance has its own operation map and
pruning is performed on a per keymaster instance basis.
* Introduce KeyBlobEntries which are independent from files. To allow
concurrent access to the key blob data base, entries can be
individually locked so that operations on entries become atomic.
LockedKeyBlobEntries are move only objects that track ownership of an
Entry on the stack or in functor object representing keymaster worker
requests. Entries must only be locked by the dispatcher Thread. Worker
threads can only be granted access to a LockedKeyBlobEntry by the
dispatcher thread. This allows the dispatcher thread to execute a
barrier that waits until all locks held by workers have been
relinquished to perform blob database maintenance operations, e.g.,
clearing a uid of all entries.
* Verification tokens are now acquired asynchronously. When a begin
operation requires a verification token a request is submitted to the
other keymaster worker while the begin call returns. When the
operation commences with update or finish, we block until the
verification token becomes available.
As of this patch the keystore IPC interface is still synchronous. That
is, the dispatcher thread dispatches a request to a worker and then
waits until the worker has finished. In a followup patch the IPC
interface shall be made asynchronous so that multiple requests may be in
flight.
Test: Ran full CTS test suite
atest android.keystore.cts
Bug: 111443219
Bug: 110495056
Change-Id: I305e28d784295a0095a34810d83202f7423498bd
diff --git a/keystore/blob.h b/keystore/blob.h
index 665e07a..5cd1b90 100644
--- a/keystore/blob.h
+++ b/keystore/blob.h
@@ -22,8 +22,14 @@
#include <openssl/aes.h>
#include <openssl/md5.h>
+#include <condition_variable>
+#include <functional>
#include <keystore/keymaster_types.h>
#include <keystore/keystore.h>
+#include <list>
+#include <mutex>
+#include <set>
+#include <sstream>
constexpr size_t kValueSize = 32768;
constexpr size_t kAesKeySize = 128 / 8;
@@ -80,27 +86,45 @@
TYPE_KEY_PAIR = 3,
TYPE_KEYMASTER_10 = 4,
TYPE_KEY_CHARACTERISTICS = 5,
+ TYPE_KEY_CHARACTERISTICS_CACHE = 6,
} BlobType;
class Entropy;
+class LockedKeyBlobEntry;
+/**
+ * The Blob represents the content of a KeyBlobEntry.
+ *
+ * BEWARE: It is only save to call any member function of a Blob b if bool(b) yields true.
+ * Exceptions are putKeyCharacteristics(), the assignment operators and operator bool.
+ */
class Blob {
+ friend LockedKeyBlobEntry;
+
public:
Blob(const uint8_t* value, size_t valueLength, const uint8_t* info, uint8_t infoLength,
BlobType type);
explicit Blob(blobv3 b);
Blob();
+ Blob(const Blob& rhs);
+ Blob(Blob&& rhs);
- ~Blob() { mBlob = {}; }
+ ~Blob() {
+ if (mBlob) *mBlob = {};
+ }
- const uint8_t* getValue() const { return mBlob.value; }
+ Blob& operator=(const Blob& rhs);
+ Blob& operator=(Blob&& rhs);
+ operator bool() const { return bool(mBlob); }
- int32_t getLength() const { return mBlob.length; }
+ const uint8_t* getValue() const { return mBlob->value; }
- const uint8_t* getInfo() const { return mBlob.value + mBlob.length; }
- uint8_t getInfoLength() const { return mBlob.info; }
+ int32_t getLength() const { return mBlob->length; }
- uint8_t getVersion() const { return mBlob.version; }
+ const uint8_t* getInfo() const { return mBlob->value + mBlob->length; }
+ uint8_t getInfoLength() const { return mBlob->info; }
+
+ uint8_t getVersion() const { return mBlob->version; }
bool isEncrypted() const;
void setEncrypted(bool encrypted);
@@ -111,22 +135,141 @@
bool isCriticalToDeviceEncryption() const;
void setCriticalToDeviceEncryption(bool critical);
- bool isFallback() const { return mBlob.flags & KEYSTORE_FLAG_FALLBACK; }
+ bool isFallback() const { return mBlob->flags & KEYSTORE_FLAG_FALLBACK; }
void setFallback(bool fallback);
- void setVersion(uint8_t version) { mBlob.version = version; }
- BlobType getType() const { return BlobType(mBlob.type); }
- void setType(BlobType type) { mBlob.type = uint8_t(type); }
+ void setVersion(uint8_t version) { mBlob->version = version; }
+ BlobType getType() const { return BlobType(mBlob->type); }
+ void setType(BlobType type) { mBlob->type = uint8_t(type); }
keystore::SecurityLevel getSecurityLevel() const;
void setSecurityLevel(keystore::SecurityLevel);
- ResponseCode writeBlob(const std::string& filename, const uint8_t* aes_key, State state,
- Entropy* entropy);
- ResponseCode readBlob(const std::string& filename, const uint8_t* aes_key, State state);
+ std::tuple<bool, keystore::AuthorizationSet, keystore::AuthorizationSet>
+ getKeyCharacteristics() const;
+
+ bool putKeyCharacteristics(const keystore::AuthorizationSet& hwEnforced,
+ const keystore::AuthorizationSet& swEnforced);
private:
- blobv3 mBlob;
+ std::unique_ptr<blobv3> mBlob;
+
+ ResponseCode readBlob(const std::string& filename, const uint8_t* aes_key, State state);
+};
+
+/**
+ * A KeyBlobEntry represents a full qualified key blob as known by Keystore. The key blob
+ * is given by the uid of the owning app and the alias used by the app to refer to this key.
+ * The user_dir_ is technically implied by the uid, but computation of the user directory is
+ * done in the user state database. Which is why we also cache it here.
+ *
+ * The KeyBlobEntry knows the location of the key blob files (which may include a characteristics
+ * cache file) but does not allow read or write access to the content. It also does not imply
+ * the existence of the files.
+ *
+ * KeyBlobEntry abstracts, to some extent, from the the file system based storage of key blobs.
+ * An evolution of KeyBlobEntry may be used for key blob storage based on a back end other than
+ * file system, e.g., SQL database or other.
+ *
+ * For access to the key blob content the programmer has to acquire a LockedKeyBlobEntry (see
+ * below).
+ */
+class KeyBlobEntry {
+ private:
+ std::string alias_;
+ std::string user_dir_;
+ uid_t uid_;
+ bool masterkey_;
+
+ public:
+ KeyBlobEntry(std::string alias, std::string user_dir, uid_t uid, bool masterkey = false)
+ : alias_(std::move(alias)), user_dir_(std::move(user_dir)), uid_(uid),
+ masterkey_(masterkey) {}
+
+ std::string getKeyBlobBaseName() const;
+ std::string getKeyBlobPath() const;
+
+ std::string getCharacteristicsBlobBaseName() const;
+ std::string getCharacteristicsBlobPath() const;
+
+ bool hasKeyBlob() const;
+ bool hasCharacteristicsBlob() const;
+
+ bool operator<(const KeyBlobEntry& rhs) const {
+ return std::tie(alias_, user_dir_, uid_) < std::tie(rhs.alias_, rhs.user_dir_, uid_);
+ }
+ bool operator==(const KeyBlobEntry& rhs) const {
+ return std::tie(alias_, user_dir_, uid_) == std::tie(rhs.alias_, rhs.user_dir_, uid_);
+ }
+ bool operator!=(const KeyBlobEntry& rhs) const { return !(*this == rhs); }
+
+ inline const std::string& alias() const { return alias_; }
+ inline const std::string& user_dir() const { return user_dir_; }
+ inline uid_t uid() const { return uid_; }
+};
+
+/**
+ * The LockedKeyBlobEntry is a proxy object to KeyBlobEntry that expresses exclusive ownership
+ * of a KeyBlobEntry. LockedKeyBlobEntries can be acquired by calling
+ * LockedKeyBlobEntry::get() or LockedKeyBlobEntry::list().
+ *
+ * LockedKeyBlobEntries are movable but not copyable. By convention they can only
+ * be taken by the dispatcher thread of keystore, but not by any keymaster worker thread.
+ * The dispatcher thread may transfer ownership of a locked entry to a keymaster worker thread.
+ *
+ * Locked entries are tracked on the stack or as members of movable functor objects passed to the
+ * keymaster worker request queues. Locks are relinquished as the locked entry gets destroyed, e.g.,
+ * when it goes out of scope or when the owning request functor gets destroyed.
+ *
+ * LockedKeyBlobEntry::list(), which must only be called by the dispatcher, blocks until all
+ * LockedKeyBlobEntries have been destroyed. Thereby list acts as a fence to make sure it gets a
+ * consistent view of the key blob database. Under the assumption that keymaster worker requests
+ * cannot run or block indefinitely and cannot grab new locked entries, progress is guaranteed.
+ * It then grabs locked entries in accordance with the given filter rule.
+ *
+ * LockedKeyBlobEntry allow access to the proxied KeyBlobEntry interface through the operator->.
+ * They add additional functionality to access and modify the key blob's content on disk.
+ * LockedKeyBlobEntry ensures atomic operations on the persistently stored key blobs on a per
+ * entry granularity.
+ */
+class LockedKeyBlobEntry {
+ private:
+ static std::set<KeyBlobEntry> locked_blobs_;
+ static std::mutex locked_blobs_mutex_;
+ static std::condition_variable locked_blobs_mutex_cond_var_;
+
+ const KeyBlobEntry* entry_;
+ LockedKeyBlobEntry(const KeyBlobEntry& entry) : entry_(&entry) {}
+
+ static void put(const KeyBlobEntry& entry);
+ LockedKeyBlobEntry(const LockedKeyBlobEntry&) = delete;
+ LockedKeyBlobEntry& operator=(const LockedKeyBlobEntry&) = delete;
+
+ public:
+ LockedKeyBlobEntry() : entry_(nullptr){};
+ ~LockedKeyBlobEntry();
+ LockedKeyBlobEntry(LockedKeyBlobEntry&& rhs) : entry_(rhs.entry_) { rhs.entry_ = nullptr; }
+ LockedKeyBlobEntry& operator=(LockedKeyBlobEntry&& rhs) {
+ // as dummy goes out of scope it relinquishes the lock on this
+ LockedKeyBlobEntry dummy(std::move(*this));
+ entry_ = rhs.entry_;
+ rhs.entry_ = nullptr;
+ return *this;
+ }
+ static LockedKeyBlobEntry get(KeyBlobEntry entry);
+ static std::tuple<ResponseCode, std::list<LockedKeyBlobEntry>>
+ list(const std::string& user_dir,
+ std::function<bool(uid_t, const std::string&)> filter =
+ [](uid_t, const std::string&) -> bool { return true; });
+
+ ResponseCode writeBlobs(Blob keyBlob, Blob characteristicsBlob, const uint8_t* aes_key,
+ State state, Entropy* entorpy) const;
+ std::tuple<ResponseCode, Blob, Blob> readBlobs(const uint8_t* aes_key, State state) const;
+ ResponseCode deleteBlobs() const;
+
+ inline operator bool() const { return entry_ != nullptr; }
+ inline const KeyBlobEntry& operator*() const { return *entry_; }
+ inline const KeyBlobEntry* operator->() const { return entry_; }
};
#endif // KEYSTORE_BLOB_H_