Add more docs for datastore library
Bug: 325144964
Test: atest SettingsLibDataStoreTest
Change-Id: I1649341f0e75ab89a71ad7037a30b27fe08c7e1c
diff --git a/packages/SettingsLib/DataStore/README.md b/packages/SettingsLib/DataStore/README.md
index 30cb993..a762ad3 100644
--- a/packages/SettingsLib/DataStore/README.md
+++ b/packages/SettingsLib/DataStore/README.md
@@ -1,55 +1,93 @@
# Datastore library
-This library aims to manage datastore in a consistent way.
+This library provides consistent API for data management (including backup,
+restore, and metrics logging) on Android platform.
+
+Notably, it is designed to be flexible and could be utilized for a wide range of
+data store besides the settings preferences.
## Overview
-A datastore is required to extend the `BackupRestoreStorage` class and implement
-either `Observable` or `KeyedObservable` interface, which enforces:
+In the high-level design, a persistent datastore aims to support two key
+characteristics:
-- Backup and restore: Datastore should support
- [data backup](https://developer.android.com/guide/topics/data/backup) to
- preserve user experiences on a new device.
-- Observer pattern: The
- [observer pattern](https://en.wikipedia.org/wiki/Observer_pattern) allows to
- monitor data change in the datastore and
- - trigger
- [BackupManager.dataChanged](https://developer.android.com/reference/android/app/backup/BackupManager#dataChanged\(\))
- automatically.
- - track data change event to log metrics.
- - update internal state and take action.
+- **observable**: triggers backup and metrics logging whenever data is
+ changed.
+- **transferable**: offers users with a seamless experience by backing up and
+ restoring data on to new devices.
+
+More specifically, Android framework supports
+[data backup](https://developer.android.com/guide/topics/data/backup) to
+preserve user experiences on a new device. And the
+[observer pattern](https://en.wikipedia.org/wiki/Observer_pattern) allows to
+monitor data change.
### Backup and restore
-The Android backup framework provides
+Currently, the Android backup framework provides
[BackupAgentHelper](https://developer.android.com/reference/android/app/backup/BackupAgentHelper)
and
[BackupHelper](https://developer.android.com/reference/android/app/backup/BackupHelper)
-to back up a datastore. However, there are several caveats when implement
-`BackupHelper`:
+to facilitate data backup. However, there are several caveats to consider when
+implementing `BackupHelper`:
-- performBackup: The data is updated incrementally but it is not well
+- *performBackup*: The data is updated incrementally but it is not well
documented. The `ParcelFileDescriptor` state parameters are normally ignored
and data is updated even there is no change.
-- restoreEntity: The implementation must take care not to seek or close the
- underlying data source, nor read more than size() bytes from the stream when
- restore (see
+- *restoreEntity*: The implementation must take care not to seek or close the
+ underlying data source, nor read more than `size()` bytes from the stream
+ when restore (see
[BackupDataInputStream](https://developer.android.com/reference/android/app/backup/BackupDataInputStream)).
- It is possible a `BackupHelper` prevents other `BackupHelper`s from
- restoring data.
-- writeNewStateDescription: Existing implementations rarely notice that this
- callback is invoked after all entities are restored, and check if necessary
- data are all restored in `restoreEntity` (e.g.
+ It is possible that a `BackupHelper` interferes with the restore process of
+ other `BackupHelper`s.
+- *writeNewStateDescription*: Existing implementations rarely notice that this
+ callback is invoked after *all* entities are restored. Instead, they check
+ if necessary data are all restored in the `restoreEntity` (e.g.
[BatteryBackupHelper](https://cs.android.com/android/platform/superproject/main/+/main:packages/apps/Settings/src/com/android/settings/fuelgauge/BatteryBackupHelper.java;l=144;drc=cca804e1ed504e2d477be1e3db00fb881ca32736)),
which is not robust sometimes.
-This library provides more clear API and offers some improvements:
+The datastore library will mitigate these problems by providing alternative
+APIs. For instance, library users make use of `InputStream` / `OutputStream` to
+back up and restore data directly.
-- The implementation only needs to focus on the `BackupRestoreEntity`
- interface. The `InputStream` of restore will ensure bounded data are read,
- and close the stream will be no-op.
-- The library computes checksum of the backup data automatically, so that
- unchanged data will not be sent to Android backup system.
+### Observer pattern
+
+In the current implementation, the Android backup framework requires a manual
+call to
+[BackupManager.dataChanged](https://developer.android.com/reference/android/app/backup/BackupManager#dataChanged\(\)).
+However, it's often observed that this API call is forgotten when using
+`SharedPreferences`. Additionally, there's a common need to log metrics when
+data changed. To address these limitations, datastore API employed the observer
+pattern.
+
+### API design and advantages
+
+Datastore must extend the `BackupRestoreStorage` class (subclass of
+[BackupHelper](https://developer.android.com/reference/android/app/backup/BackupHelper)).
+The data in a datastore is group by entity, which is represented by
+`BackupRestoreEntity`. Basically, a datastore implementation only needs to focus
+on the `BackupRestoreEntity`.
+
+If the datastore is key-value based (e.g. `SharedPreferences`), implements the
+`KeyedObservable` interface to offer fine-grained observer. Otherwise,
+implements `Observable`. There are builtin thread-safe implementations of the
+two interfaces (`KeyedDataObservable` / `DataObservable`). If it is Kotlin, use
+delegation to simplify the code.
+
+Keep in mind that the implementation should call `KeyedObservable.notifyChange`
+/ `Observable.notifyChange` whenever internal data is changed, so that the
+registered observer will be notified properly.
+
+For `SharedPreferences` use case, leverage the `SharedPreferencesStorage`
+directly. To back up other file based storage, extend the
+`BackupRestoreFileStorage` class.
+
+Here are some highlights of the library:
+
+- The restore `InputStream` will ensure bounded data are read, and close the
+ stream is no-op. That being said, all entities are isolated.
+- Data checksum is computed automatically, unchanged data will not be sent to
+ Android backup system.
- Data compression is supported:
- ZIP best compression is enabled by default, no extra effort needs to be
taken.
@@ -67,98 +105,159 @@
successfully restored in those older versions. This is achieved by extending
the `BackupRestoreFileStorage` class, and `BackupRestoreFileArchiver` will
treat each file as an entity and do the backup / restore.
-- Manual `BackupManager.dataChanged` call is unnecessary now, the library will
- do the invocation (see next section).
+- Manual `BackupManager.dataChanged` call is unnecessary now, the framework
+ will invoke the API automatically.
-### Observer pattern
+## Usages
-Manual `BackupManager.dataChanged` call is required by current backup framework.
-In practice, it is found that `SharedPreferences` usages foget to invoke the
-API. Besides, there are common use cases to log metrics when data is changed.
-Consequently, observer pattern is employed to resolve the issues.
+This section provides [examples](example/ExampleStorage.kt) of datastore.
-If the datastore is key-value based (e.g. `SharedPreferences`), implements the
-`KeyedObservable` interface to offer fine-grained observer. Otherwise,
-implements `Observable`. The library provides thread-safe implementations
-(`KeyedDataObservable` / `DataObservable`), and Kotlin delegation will be
-helpful.
-
-Keep in mind that the implementation should call `KeyedObservable.notifyChange`
-/ `Observable.notifyChange` whenever internal data is changed, so that the
-registered observer will be notified properly.
-
-## Usage and example
-
-For `SharedPreferences` use case, leverage the `SharedPreferencesStorage`. To
-back up other file based storage, extend the `BackupRestoreFileStorage` class.
-
-Here is an example of customized datastore, which has a string to back up:
+Here is a datastore with a string data:
```kotlin
-class MyDataStore : ObservableBackupRestoreStorage() {
- // Another option is make it a StringEntity type and maintain a String field inside StringEntity
- @Volatile // backup/restore happens on Binder thread
- var data: String? = null
- private set
+class ExampleStorage : ObservableBackupRestoreStorage() {
+ @Volatile // field is manipulated by multiple threads, synchronization might be needed
+ var data: String? = null
+ private set
- fun setData(data: String?) {
- this.data = data
- notifyChange(ChangeReason.UPDATE)
+ @AnyThread
+ fun setData(data: String?) {
+ this.data = data
+ // call notifyChange to trigger backup and metrics logging whenever data is changed
+ if (data != null) {
+ notifyChange(ChangeReason.UPDATE)
+ } else {
+ notifyChange(ChangeReason.DELETE)
}
-
- override val name: String
- get() = "MyData"
-
- override fun createBackupRestoreEntities(): List<BackupRestoreEntity> =
- listOf(StringEntity("data"))
-
- private inner class StringEntity(override val key: String) : BackupRestoreEntity {
- override fun backup(
- backupContext: BackupContext,
- outputStream: OutputStream,
- ) =
- if (data != null) {
- outputStream.write(data!!.toByteArray(UTF_8))
- EntityBackupResult.UPDATE
- } else {
- EntityBackupResult.DELETE
- }
-
- override fun restore(restoreContext: RestoreContext, inputStream: InputStream) {
- data = String(inputStream.readAllBytes(), UTF_8)
- // NOTE: The library will call notifyChange(ChangeReason.RESTORE) for you
- }
- }
-
- override fun onRestoreFinished() {
- // TODO: Update state with the restored data. Use this callback instead "restore()" in case
- // the restore action involves several entities.
- // NOTE: The library will call notifyChange(ChangeReason.RESTORE) for you
- }
-}
-```
-
-In the application class:
-
-```kotlin
-class MyApplication : Application() {
- override fun onCreate() {
- super.onCreate();
- BackupRestoreStorageManager.getInstance(this).add(MyDataStore());
}
-}
-```
-In the custom `BackupAgentHelper` class:
+ override val name: String
+ get() = "ExampleStorage"
-```kotlin
-class MyBackupAgentHelper : BackupAgentHelper() {
- override fun onCreate() {
- BackupRestoreStorageManager.getInstance(this).addBackupAgentHelpers(this);
+ override fun createBackupRestoreEntities(): List<BackupRestoreEntity> =
+ listOf(StringEntity("data"))
+
+ override fun enableRestore(): Boolean {
+ return true // check condition like flag, environment, etc.
+ }
+
+ override fun enableBackup(backupContext: BackupContext): Boolean {
+ return true // check condition like flag, environment, etc.
+ }
+
+ @BinderThread
+ private inner class StringEntity(override val key: String) : BackupRestoreEntity {
+ override fun backup(backupContext: BackupContext, outputStream: OutputStream) =
+ if (data != null) {
+ outputStream.write(data!!.toByteArray(UTF_8))
+ EntityBackupResult.UPDATE
+ } else {
+ EntityBackupResult.DELETE // delete existing backup data
+ }
+
+ override fun restore(restoreContext: RestoreContext, inputStream: InputStream) {
+ // DO NOT call setData API here, which will trigger notifyChange unexpectedly.
+ // Under the hood, the datastore library will call notifyChange(ChangeReason.RESTORE)
+ // later to notify observers.
+ data = String(inputStream.readBytes(), UTF_8)
+ // Handle restored data in onRestoreFinished() callback
+ }
}
override fun onRestoreFinished() {
- BackupRestoreStorageManager.getInstance(this).onRestoreFinished();
+ // TODO: Update state with the restored data. Use this callback instead of "restore()" in
+ // case the restore action involves several entities.
+ // NOTE: The library will call notifyChange(ChangeReason.RESTORE) for you
}
}
```
+
+And this is a datastore with key value data:
+
+```kotlin
+class ExampleKeyValueStorage :
+ BackupRestoreStorage(), KeyedObservable<String> by KeyedDataObservable() {
+ // thread safe data structure
+ private val map = ConcurrentHashMap<String, String>()
+
+ override val name: String
+ get() = "ExampleKeyValueStorage"
+
+ fun updateData(key: String, value: String?) {
+ if (value != null) {
+ map[key] = value
+ notifyChange(ChangeReason.UPDATE)
+ } else {
+ map.remove(key)
+ notifyChange(ChangeReason.DELETE)
+ }
+ }
+
+ override fun createBackupRestoreEntities(): List<BackupRestoreEntity> =
+ listOf(createMapBackupRestoreEntity())
+
+ private fun createMapBackupRestoreEntity() =
+ object : BackupRestoreEntity {
+ override val key: String
+ get() = "map"
+
+ override fun backup(
+ backupContext: BackupContext,
+ outputStream: OutputStream,
+ ): EntityBackupResult {
+ // Use TreeMap to achieve predictable and stable order, so that data will not be
+ // updated to Android backup backend if there is only order change.
+ val copy = TreeMap(map)
+ if (copy.isEmpty()) return EntityBackupResult.DELETE
+ val dataOutputStream = DataOutputStream(outputStream)
+ dataOutputStream.writeInt(copy.size)
+ for ((key, value) in copy) {
+ dataOutputStream.writeUTF(key)
+ dataOutputStream.writeUTF(value)
+ }
+ return EntityBackupResult.UPDATE
+ }
+
+ override fun restore(restoreContext: RestoreContext, inputStream: InputStream) {
+ val dataInputString = DataInputStream(inputStream)
+ repeat(dataInputString.readInt()) {
+ val key = dataInputString.readUTF()
+ val value = dataInputString.readUTF()
+ map[key] = value
+ }
+ }
+ }
+}
+```
+
+All the datastore should be added in the application class:
+
+```kotlin
+class ExampleApplication : Application() {
+ override fun onCreate() {
+ super.onCreate()
+ BackupRestoreStorageManager.getInstance(this)
+ .add(ExampleStorage(), ExampleKeyValueStorage())
+ }
+}
+```
+
+Additionally, inject datastore to the custom `BackupAgentHelper` class:
+
+```kotlin
+class ExampleBackupAgent : BackupAgentHelper() {
+ override fun onCreate() {
+ super.onCreate()
+ BackupRestoreStorageManager.getInstance(this).addBackupAgentHelpers(this)
+ }
+
+ override fun onRestoreFinished() {
+ BackupRestoreStorageManager.getInstance(this).onRestoreFinished()
+ }
+}
+```
+
+## Development
+
+Please preserve the code coverage ratio during development. The current line
+coverage is **100% (444/444)** and branch coverage is **93.6% (176/188)**.
diff --git a/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreEntity.kt b/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreEntity.kt
index 817ee4c..6720e5c 100644
--- a/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreEntity.kt
+++ b/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreEntity.kt
@@ -23,7 +23,11 @@
import java.io.InputStream
import java.io.OutputStream
-/** Entity for back up and restore. */
+/**
+ * Entity for back up and restore.
+ *
+ * Note that backup/restore callback is invoked on the binder thread.
+ */
interface BackupRestoreEntity {
/**
* Key of the entity.
@@ -45,9 +49,12 @@
/**
* Backs up the entity.
*
+ * Back up data in predictable order (e.g. use `TreeMap` instead of `HashMap`), otherwise data
+ * will be backed up needlessly.
+ *
* @param backupContext context for backup
* @param outputStream output stream to back up data
- * @return false if backup file is deleted, otherwise true
+ * @return backup result
*/
@BinderThread
@Throws(IOException::class)
diff --git a/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreStorage.kt b/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreStorage.kt
index 935f9cc..284c97b 100644
--- a/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreStorage.kt
+++ b/packages/SettingsLib/DataStore/src/com/android/settingslib/datastore/BackupRestoreStorage.kt
@@ -22,6 +22,7 @@
import android.app.backup.BackupHelper
import android.os.ParcelFileDescriptor
import android.util.Log
+import androidx.annotation.BinderThread
import androidx.annotation.VisibleForTesting
import androidx.collection.MutableScatterMap
import com.google.common.io.ByteStreams
@@ -38,16 +39,22 @@
import java.util.zip.CheckedInputStream
import java.util.zip.CheckedOutputStream
import java.util.zip.Checksum
+import javax.annotation.concurrent.ThreadSafe
internal const val LOG_TAG = "BackupRestoreStorage"
/**
- * Storage with backup and restore support. Subclass must implement either [Observable] or
- * [KeyedObservable] interface.
+ * Storage with backup and restore support.
+ *
+ * Subclass MUST
+ * - implement either [Observable] or [KeyedObservable] interface.
+ * - be thread safe, backup/restore happens on Binder thread, while general data read/write
+ * operations occur on other threads.
*
* The storage is identified by a unique string [name] and data set is split into entities
* ([BackupRestoreEntity]).
*/
+@ThreadSafe
abstract class BackupRestoreStorage : BackupHelper {
/**
* A unique string used to disambiguate the various storages within backup agent.
@@ -68,7 +75,7 @@
@VisibleForTesting internal var entities: List<BackupRestoreEntity>? = null
/** Entities to back up and restore. */
- abstract fun createBackupRestoreEntities(): List<BackupRestoreEntity>
+ @BinderThread abstract fun createBackupRestoreEntities(): List<BackupRestoreEntity>
/** Default codec used to encode/decode the entity data. */
open fun defaultCodec(): BackupCodec = BackupZipCodec.BEST_COMPRESSION
@@ -134,7 +141,11 @@
Log.i(LOG_TAG, "[$name] Backup end")
}
- /** Returns if backup is enabled. */
+ /**
+ * Returns if backup is enabled.
+ *
+ * If disabled, [performBackup] will be no-op, all entities backup are skipped.
+ */
open fun enableBackup(backupContext: BackupContext): Boolean = true
open fun wrapBackupOutputStream(codec: BackupCodec, outputStream: OutputStream): OutputStream {
@@ -172,7 +183,11 @@
private fun ensureEntities(): List<BackupRestoreEntity> =
entities ?: createBackupRestoreEntities().also { entities = it }
- /** Returns if restore is enabled. */
+ /**
+ * Returns if restore is enabled.
+ *
+ * If disabled, [restoreEntity] will be no-op, all entities restore are skipped.
+ */
open fun enableRestore(): Boolean = true
open fun wrapRestoreInputStream(
@@ -188,12 +203,13 @@
}
final override fun writeNewStateDescription(newState: ParcelFileDescriptor) {
+ if (!enableRestore()) return
entities = null // clear to reduce memory footprint
newState.writeAndClearEntityStates()
onRestoreFinished()
}
- /** Callbacks when restore finished. */
+ /** Callbacks when entity data are all restored. */
open fun onRestoreFinished() {}
@VisibleForTesting
diff --git a/packages/SettingsLib/DataStore/tests/src/com/android/settingslib/datastore/BackupRestoreStorageTest.kt b/packages/SettingsLib/DataStore/tests/src/com/android/settingslib/datastore/BackupRestoreStorageTest.kt
index 99998ff..26534ba 100644
--- a/packages/SettingsLib/DataStore/tests/src/com/android/settingslib/datastore/BackupRestoreStorageTest.kt
+++ b/packages/SettingsLib/DataStore/tests/src/com/android/settingslib/datastore/BackupRestoreStorageTest.kt
@@ -248,6 +248,15 @@
}
@Test
+ fun writeNewStateDescription_restoreDisabled() {
+ val storage = spy(TestStorage().apply { enabled = false })
+ temporaryFolder.newFile().toParcelFileDescriptor(MODE_WRITE_ONLY or MODE_APPEND).use {
+ storage.writeNewStateDescription(it)
+ }
+ verify(storage, never()).onRestoreFinished()
+ }
+
+ @Test
fun backupAndRestore() {
val storage = spy(TestStorage(entity1, entity2))
val backupAgentHelper = BackupAgentHelper()