Merge "pvmfw: Zero all scratch memory before guest runs"
diff --git a/docs/getting_started/index.md b/docs/getting_started/index.md
index 25eb909..62529b3 100644
--- a/docs/getting_started/index.md
+++ b/docs/getting_started/index.md
@@ -11,8 +11,8 @@
We support the following device:
* aosp_cf_x86_64_phone (Cuttlefish a.k.a. Cloud Android)
-* oriole (Pixel 6)
-* raven (Pixel 6 Pro)
+* oriole/raven (Pixel 6, and 6 Pro)
+* panther/cheetah (Pixel 7, and 7 Pro)
### Cuttlefish
@@ -30,12 +30,14 @@
acloud create --local-instance --local-image
```
-### Pixel 6 and 6 Pro
+### Google Pixel phones
-If the device is running Android 12, join the [Android Beta
-Program](https://www.google.com/android/beta) to upgrade to Android 13 Beta.
+If the device is running Android 13 or earlier, join the [Android Beta
+Program](https://developer.android.com/about/versions/14/get#on_pixel) to upgrade to Android 14
+Beta.
-Once upgraded to Android 13, execute the following command to enable pKVM.
+Once upgraded to Android 14, and if you are using Pixel 6 or 6 Pro, execute the following command to
+enable pKVM. You don't need to do this for Pixel 7 and 7 Pro.
```shell
adb reboot bootloader
@@ -44,41 +46,6 @@
fastboot reboot
```
-Due to a bug in Android 13 for these devices, pKVM may stop working after an
-[OTA update](https://source.android.com/devices/tech/ota). To prevent this, it
-is necessary to manually replicate the `pvmfw` partition to the other slot:
-
-```shell
-git -C <android_root>/packages/modules/Virtualization
-show de6b0b2ecf6225a0a7b43241de27e74fc3e6ceb2:pvmfw/pvmfw.img > /tmp/pvmfw.img
-adb reboot bootloader
-fastboot --slot other flash pvmfw /tmp/pvmfw.img
-fastboot reboot
-```
-
-Otherwise, if an OTA has already made pKVM unusable, the working partition
-should be copied to the "current" slot:
-
-```shell
-adb reboot bootloader
-fastboot flash pvmfw /tmp/pvmfw.img
-fastboot reboot
-```
-
-Finally, if the `pvmfw` partition has been corrupted, both slots may be flashed
-using the [`pvmfw.img` pre-built](https://android.googlesource.com/platform/packages/modules/Virtualization/+/08deac98acefd62e222edfa374d5292458cf97eb%5E/pvmfw/pvmfw.img)
-as long as the bootloader remains unlocked. Otherwise, a fresh install of
-Android 13 followed by the manual steps above for flashing the `other` slot
-should be used as a last resort.
-
-Starting in Android 14, `pvmfw.img` can be built using the Android Build system:
-```
-lunch <target> # where PRODUCT_BUILD_PVMFW_IMAGE=true
-m pvmfwimage # partition image under ${ANDROID_PRODUCT_OUT}/pvmfw.img
-```
-Note that the result is not intended to be used in Android 13 as not
-backward-compatibility is guaranteed.
-
## Running demo app
The instruction is [here](../../demo/README.md).
@@ -140,23 +107,6 @@
adb shell /apex/com.android.virt/bin/vm run-microdroid --debug full
```
-The `instance.img` and `apk.idsig` files will be stored in a subdirectory under
-`/data/local/tmp/microdroid`, that `vm` will create.
-
-Atlernatively, you can manually run the demo app on top of Microdroid as follows:
-
-```shell
-UNBUNDLED_BUILD_SDKS_FROM_SOURCE=true TARGET_BUILD_APPS=MicrodroidDemoApp m apps_only dist
-adb shell mkdir -p /data/local/tmp/virt
-adb push out/dist/MicrodroidDemoApp.apk /data/local/tmp/virt/
-adb shell /apex/com.android.virt/bin/vm run-app \
- --debug full \
- /data/local/tmp/virt/MicrodroidDemoApp.apk \
- /data/local/tmp/virt/MicrodroidDemoApp.apk.idsig \
- /data/local/tmp/virt/instance.img \
- --payload-path MicrodroidTestNativeLib.so
-```
-
## Building and updating CrosVM and VirtualizationService {#building-and-updating}
You can update CrosVM and the VirtualizationService by updating the `com.android.virt` APEX instead
diff --git a/docs/getting_started/pixel6.md b/docs/getting_started/pixel6.md
deleted file mode 100644
index 82391c4..0000000
--- a/docs/getting_started/pixel6.md
+++ /dev/null
@@ -1,129 +0,0 @@
-# Instructions for building custom AVF on Pixel 6 or 6 Pro
-
-This document provides steps for building AVF from AOSP, and then install it to
-Pixel 6 series to better understand AVF and do some experiments.
-
-**WARNING**: Unless Android 13 is released to AOSP (expected to be at Summer
-2022, exact date TBD) by the time when you read this documentation, or you or
-your company have early access to Android Tiramisu source tree, you **CANNOT**
-follow this instruction. In that case, you can only **USE** the AVF that is
-shipped in the Android 13 Beta Image.
-
-This is because AVF in the beta image is signed by Google and therefore it can't
-be updated to a new AVF built in AOSP which can't be signed by the Google key
-that is not shared with AOSP.
-
-## Upgrade to Android 13 Beta Image
-
-First, upgrade your Pixel 6 or Pixel 6 Pro to the Android 13 Beta Image. This
-can be done in two ways:
-
-* Join [Android Beta Program](https://www.google.com/android/beta) and then OTA
- to Android 13.
-* Manually flash [Android 13 Beta Images](https://developer.android.com/about/versions/13/download#factory-images).
-
-Then enable ADB debugging in "Settings" -> "System" -> "Developer options".
-Finally, enable PKVM.
-
-```shell
-adb reboot bootloader
-fastboot flashing unlock
-fastboot oem pkvm enable
-fastboot reboot
-```
-
-## Building GSI and flashing it
-
-Prepare your Android 13 (Tiramisu) source tree.
-
-```shell
-mkdir tm
-cd tm
-repo init -u <URL> -m <your_tm_branch>
-repo sync -c --no-tags -j 10
-```
-
-Patch GSI so that it includes AVF. Edit
-`build/make/target/product/gsi_release.mk` and add the following line to the
-end (or anywhere in the file that makes sense):
-
-```
-PRODUCT_PACKAGES += com.android.virt
-```
-
-Build GSI.
-
-```shell
-source build/envsetup.sh
-choosecombo 1 aosp_arm64 userdebug
-m
-```
-
-Flash GSI to the Pixel device.
-
-```shell
-adb reboot bootloader
-fastboot reboot fastboot
-fastboot delete-logical-partition product_a
-fastboot flash system out/target/product/generic_arm64/system.img
-fastboot --disable-verification flash vbmeta out/target/product/generic_arm64/vbmeta.img
-fastboot -w reboot
-```
-
-Deleting the logical partition `product_a` is needed because the GSI image is
-bigger than the logical partition `system_a` of the beta image.
-`--disable-verification` when flashing the `vbmeta` partition is critical. Don't
-miss it.
-
-Lastly, check if you are running GSI.
-
-```shell
-adb shell getprop ro.build.product
-adb shell ls /dev/kvm
-adb shell ls /apex/com.android.virt/bin/vm
-```
-
-The result should be as follows.
-
-```
-generic_arm64
-/dev/kvm
-/apex/com.android.virt/bin/vm
-```
-
-## Building and installing AVF from AOSP
-
-Checkout AOSP master branch.
-
-```shell
-mkdir aosp
-cd aosp
-repo init -u https://android.googlesource.com/platform/manifest -b master
-repo sync -c --no-tags -j 10
-```
-
-Then build the `com.android.virt` APEX.
-
-```shell
-source build/envsetup.sh
-banchan com.android.virt aosp_arm64
-UNBUNDLED_BUILD_SDKS_FROM_SOURCE=true m apps_only dist
-```
-
-Install the newly built AVF to the device
-
-```shell
-adb install out/dist/com.android.virt.apex
-adb reboot
-```
-
-If this doesn't work for some reason, try this:
-
-```
-adb root
-adb shell setenforce 0
-adb push out/dist/com.android.virt.apex /data/local/
-adb shell cmd -w apexservice deactivatePackage /system/system_ext/apex/com.android.virt.apex
-adb shell cmd -w apexservice activatePackage /data/local/com.android.virt.apex
-// Don't adb reboot
-```
diff --git a/libs/hyp/Android.bp b/libs/hyp/Android.bp
index e4353c8..1d572e5 100644
--- a/libs/hyp/Android.bp
+++ b/libs/hyp/Android.bp
@@ -8,7 +8,10 @@
srcs: ["src/lib.rs"],
prefer_rlib: true,
rustlibs: [
+ "libbitflags",
+ "libonce_cell_nostd",
"libpsci",
+ "libuuid_nostd",
],
no_stdlibs: true,
stdlibs: [
diff --git a/libs/hyp/src/error.rs b/libs/hyp/src/error.rs
index 4e25e7f..408150e 100644
--- a/libs/hyp/src/error.rs
+++ b/libs/hyp/src/error.rs
@@ -16,6 +16,7 @@
use crate::KvmError;
use core::{fmt, result};
+use uuid::Uuid;
/// Result type with hypervisor error.
pub type Result<T> = result::Result<T, Error>;
@@ -27,6 +28,8 @@
MmioGuardNotsupported,
/// Failed to invoke a certain KVM HVC function.
KvmError(KvmError, u32),
+ /// Unsupported Hypervisor.
+ UnsupportedHypervisorUuid(Uuid),
/// The MMIO_GUARD granule used by the hypervisor is not supported.
UnsupportedMmioGuardGranule(usize),
}
@@ -38,6 +41,9 @@
Self::KvmError(e, function_id) => {
write!(f, "Failed to invoke the HVC function with function ID {function_id}: {e}")
}
+ Self::UnsupportedHypervisorUuid(u) => {
+ write!(f, "Unsupported Hypervisor UUID {u}")
+ }
Self::UnsupportedMmioGuardGranule(g) => {
write!(f, "Unsupported MMIO guard granule: {g}")
}
diff --git a/libs/hyp/src/hypervisor/common.rs b/libs/hyp/src/hypervisor/common.rs
index 87d35b2..accef72 100644
--- a/libs/hyp/src/hypervisor/common.rs
+++ b/libs/hyp/src/hypervisor/common.rs
@@ -15,6 +15,15 @@
//! This module regroups some common traits shared by all the hypervisors.
use crate::error::Result;
+use bitflags::bitflags;
+
+bitflags! {
+ /// Capabilities that Hypervisor backends can declare support for.
+ pub struct HypervisorCap: u32 {
+ /// Capability for guest to share its memory with host at runtime.
+ const DYNAMIC_MEM_SHARE = 0b1;
+ }
+}
/// Trait for the hypervisor.
pub trait Hypervisor {
@@ -43,4 +52,7 @@
/// Returns the memory protection granule size in bytes.
fn memory_protection_granule(&self) -> Result<usize>;
+
+ /// Check if required capabilities are supported.
+ fn has_cap(&self, cap: HypervisorCap) -> bool;
}
diff --git a/libs/hyp/src/hypervisor/gunyah.rs b/libs/hyp/src/hypervisor/gunyah.rs
new file mode 100644
index 0000000..b335c87
--- /dev/null
+++ b/libs/hyp/src/hypervisor/gunyah.rs
@@ -0,0 +1,40 @@
+use super::common::{Hypervisor, HypervisorCap};
+use crate::error::Result;
+use crate::util::SIZE_4KB;
+use uuid::{uuid, Uuid};
+
+pub(super) struct GunyahHypervisor;
+
+impl GunyahHypervisor {
+ pub const UUID: Uuid = uuid!("c1d58fcd-a453-5fdb-9265-ce36673d5f14");
+}
+
+impl Hypervisor for GunyahHypervisor {
+ fn mmio_guard_init(&self) -> Result<()> {
+ Ok(())
+ }
+
+ fn mmio_guard_map(&self, _addr: usize) -> Result<()> {
+ Ok(())
+ }
+
+ fn mmio_guard_unmap(&self, _addr: usize) -> Result<()> {
+ Ok(())
+ }
+
+ fn mem_share(&self, _base_ipa: u64) -> Result<()> {
+ unimplemented!();
+ }
+
+ fn mem_unshare(&self, _base_ipa: u64) -> Result<()> {
+ unimplemented!();
+ }
+
+ fn memory_protection_granule(&self) -> Result<usize> {
+ Ok(SIZE_4KB)
+ }
+
+ fn has_cap(&self, _cap: HypervisorCap) -> bool {
+ false
+ }
+}
diff --git a/libs/hyp/src/hypervisor/kvm.rs b/libs/hyp/src/hypervisor/kvm.rs
index c0c1ac9..772160e 100644
--- a/libs/hyp/src/hypervisor/kvm.rs
+++ b/libs/hyp/src/hypervisor/kvm.rs
@@ -14,7 +14,7 @@
//! Wrappers around calls to the KVM hypervisor.
-use super::common::Hypervisor;
+use super::common::{Hypervisor, HypervisorCap};
use crate::error::{Error, Result};
use crate::util::{page_address, SIZE_4KB};
use core::fmt::{self, Display, Formatter};
@@ -22,6 +22,7 @@
error::{positive_or_error_64, success_or_error_32, success_or_error_64},
hvc64,
};
+use uuid::{uuid, Uuid};
/// Error from a KVM HVC call.
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
@@ -71,6 +72,13 @@
pub(super) struct KvmHypervisor;
+impl KvmHypervisor {
+ // Based on ARM_SMCCC_VENDOR_HYP_UID_KVM_REG values listed in Linux kernel source:
+ // https://github.com/torvalds/linux/blob/master/include/linux/arm-smccc.h
+ pub(super) const UUID: Uuid = uuid!("28b46fb6-2ec5-11e9-a9ca-4b564d003a74");
+ const CAPABILITIES: HypervisorCap = HypervisorCap::DYNAMIC_MEM_SHARE;
+}
+
impl Hypervisor for KvmHypervisor {
fn mmio_guard_init(&self) -> Result<()> {
mmio_guard_enroll()?;
@@ -122,6 +130,10 @@
let granule = checked_hvc64(ARM_SMCCC_KVM_FUNC_HYP_MEMINFO, args)?;
Ok(granule.try_into().unwrap())
}
+
+ fn has_cap(&self, cap: HypervisorCap) -> bool {
+ Self::CAPABILITIES.contains(cap)
+ }
}
fn mmio_guard_granule() -> Result<usize> {
diff --git a/libs/hyp/src/hypervisor/mod.rs b/libs/hyp/src/hypervisor/mod.rs
index a694029..05c82dc 100644
--- a/libs/hyp/src/hypervisor/mod.rs
+++ b/libs/hyp/src/hypervisor/mod.rs
@@ -14,28 +14,82 @@
//! Wrappers around hypervisor back-ends.
+extern crate alloc;
+
mod common;
+mod gunyah;
mod kvm;
+use crate::error::{Error, Result};
+use alloc::boxed::Box;
pub use common::Hypervisor;
+pub use common::HypervisorCap;
+use gunyah::GunyahHypervisor;
pub use kvm::KvmError;
use kvm::KvmHypervisor;
-
-static HYPERVISOR: HypervisorBackend = HypervisorBackend::Kvm;
+use once_cell::race::OnceBox;
+use psci::smccc::hvc64;
+use uuid::Uuid;
enum HypervisorBackend {
Kvm,
+ Gunyah,
}
impl HypervisorBackend {
fn get_hypervisor(&self) -> &'static dyn Hypervisor {
match self {
Self::Kvm => &KvmHypervisor,
+ Self::Gunyah => &GunyahHypervisor,
}
}
}
+impl TryFrom<Uuid> for HypervisorBackend {
+ type Error = Error;
+
+ fn try_from(uuid: Uuid) -> Result<HypervisorBackend> {
+ match uuid {
+ GunyahHypervisor::UUID => Ok(HypervisorBackend::Gunyah),
+ KvmHypervisor::UUID => Ok(HypervisorBackend::Kvm),
+ u => Err(Error::UnsupportedHypervisorUuid(u)),
+ }
+ }
+}
+
+const ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID: u32 = 0x8600ff01;
+
+fn query_vendor_hyp_call_uid() -> Uuid {
+ let args = [0u64; 17];
+ let res = hvc64(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, args);
+
+ // KVM's UUID of "28b46fb6-2ec5-11e9-a9ca-4b564d003a74" is generated by
+ // Uuid::from_u128() from an input value of
+ // 0x28b46fb6_2ec511e9_a9ca4b56_4d003a74. ARM's SMC calling convention
+ // (Document number ARM DEN 0028E) describes the UUID register mapping such
+ // that W0 contains bytes 0..3 of UUID, with byte 0 in lower order bits. In
+ // the KVM example, byte 0 of KVM's UUID (0x28) will be returned in the low
+ // 8-bits of W0, while byte 15 (0x74) will be returned in bits 31-24 of W3.
+ //
+ // `uuid` value derived below thus need to be byte-reversed before
+ // being used in Uuid::from_u128(). Alternately use Uuid::from_u128_le()
+ // to achieve the same.
+
+ let uuid = ((res[3] as u32 as u128) << 96)
+ | ((res[2] as u32 as u128) << 64)
+ | ((res[1] as u32 as u128) << 32)
+ | (res[0] as u32 as u128);
+
+ Uuid::from_u128_le(uuid)
+}
+
+fn detect_hypervisor() -> HypervisorBackend {
+ query_vendor_hyp_call_uid().try_into().expect("Unknown hypervisor")
+}
+
/// Gets the hypervisor singleton.
pub fn get_hypervisor() -> &'static dyn Hypervisor {
- HYPERVISOR.get_hypervisor()
+ static HYPERVISOR: OnceBox<HypervisorBackend> = OnceBox::new();
+
+ HYPERVISOR.get_or_init(|| Box::new(detect_hypervisor())).get_hypervisor()
}
diff --git a/libs/hyp/src/lib.rs b/libs/hyp/src/lib.rs
index 6db6ba8..694f957 100644
--- a/libs/hyp/src/lib.rs
+++ b/libs/hyp/src/lib.rs
@@ -21,4 +21,4 @@
mod util;
pub use error::{Error, Result};
-pub use hypervisor::{get_hypervisor, Hypervisor, KvmError};
+pub use hypervisor::{get_hypervisor, Hypervisor, HypervisorCap, KvmError};
diff --git a/pvmfw/src/entry.rs b/pvmfw/src/entry.rs
index 4889c28..3897c1f 100644
--- a/pvmfw/src/entry.rs
+++ b/pvmfw/src/entry.rs
@@ -28,7 +28,7 @@
use core::num::NonZeroUsize;
use core::ops::Range;
use core::slice;
-use hyp::get_hypervisor;
+use hyp::{get_hypervisor, HypervisorCap};
use log::debug;
use log::error;
use log::info;
@@ -112,6 +112,18 @@
RebootReason::InvalidFdt
})?;
+ if !get_hypervisor().has_cap(HypervisorCap::DYNAMIC_MEM_SHARE) {
+ let range = info.swiotlb_info.fixed_range().ok_or_else(|| {
+ error!("Pre-shared pool range not specified in swiotlb node");
+ RebootReason::InvalidFdt
+ })?;
+
+ memory.init_shared_pool(range).map_err(|e| {
+ error!("Failed to initialize pre-shared pool {e}");
+ RebootReason::InvalidFdt
+ })?;
+ }
+
let kernel_range = if let Some(r) = info.kernel_range {
memory.alloc_range(&r).map_err(|e| {
error!("Failed to obtain the kernel range with DT range: {e}");
diff --git a/pvmfw/src/fdt.rs b/pvmfw/src/fdt.rs
index c68fc6d..bda35a7 100644
--- a/pvmfw/src/fdt.rs
+++ b/pvmfw/src/fdt.rs
@@ -17,6 +17,7 @@
use crate::bootargs::BootArgsIterator;
use crate::cstr;
use crate::helpers::flatten;
+use crate::helpers::RangeExt as _;
use crate::helpers::GUEST_PAGE_SIZE;
use crate::helpers::SIZE_4KB;
use crate::memory::BASE_ADDR;
@@ -435,40 +436,79 @@
}
#[derive(Debug)]
-struct SwiotlbInfo {
- size: u64,
- align: u64,
+pub struct SwiotlbInfo {
+ addr: Option<usize>,
+ size: usize,
+ align: usize,
+}
+
+impl SwiotlbInfo {
+ pub fn fixed_range(&self) -> Option<Range<usize>> {
+ self.addr.map(|addr| addr..addr + self.size)
+ }
}
fn read_swiotlb_info_from(fdt: &Fdt) -> libfdt::Result<SwiotlbInfo> {
let node =
fdt.compatible_nodes(cstr!("restricted-dma-pool"))?.next().ok_or(FdtError::NotFound)?;
- let size = node.getprop_u64(cstr!("size"))?.ok_or(FdtError::NotFound)?;
- let align = node.getprop_u64(cstr!("alignment"))?.ok_or(FdtError::NotFound)?;
- Ok(SwiotlbInfo { size, align })
+ let align =
+ node.getprop_u64(cstr!("alignment"))?.ok_or(FdtError::NotFound)?.try_into().unwrap();
+
+ let (addr, size) = if let Some(mut reg) = node.reg()? {
+ let reg = reg.next().ok_or(FdtError::NotFound)?;
+ let size = reg.size.ok_or(FdtError::NotFound)?;
+ reg.addr.checked_add(size).ok_or(FdtError::BadValue)?;
+ (Some(reg.addr.try_into().unwrap()), size.try_into().unwrap())
+ } else {
+ let size = node.getprop_u64(cstr!("size"))?.ok_or(FdtError::NotFound)?.try_into().unwrap();
+ (None, size)
+ };
+
+ Ok(SwiotlbInfo { addr, size, align })
}
-fn validate_swiotlb_info(swiotlb_info: &SwiotlbInfo) -> Result<(), RebootReason> {
+fn validate_swiotlb_info(
+ swiotlb_info: &SwiotlbInfo,
+ memory: &Range<usize>,
+) -> Result<(), RebootReason> {
let size = swiotlb_info.size;
let align = swiotlb_info.align;
- if size == 0 || (size % GUEST_PAGE_SIZE as u64) != 0 {
+ if size == 0 || (size % GUEST_PAGE_SIZE) != 0 {
error!("Invalid swiotlb size {:#x}", size);
return Err(RebootReason::InvalidFdt);
}
- if (align % GUEST_PAGE_SIZE as u64) != 0 {
+ if (align % GUEST_PAGE_SIZE) != 0 {
error!("Invalid swiotlb alignment {:#x}", align);
return Err(RebootReason::InvalidFdt);
}
+
+ if let Some(range) = swiotlb_info.fixed_range() {
+ if !range.is_within(memory) {
+ error!("swiotlb range {range:#x?} not part of memory range {memory:#x?}");
+ return Err(RebootReason::InvalidFdt);
+ }
+ }
+
Ok(())
}
fn patch_swiotlb_info(fdt: &mut Fdt, swiotlb_info: &SwiotlbInfo) -> libfdt::Result<()> {
let mut node =
fdt.root_mut()?.next_compatible(cstr!("restricted-dma-pool"))?.ok_or(FdtError::NotFound)?;
- node.setprop_inplace(cstr!("size"), &swiotlb_info.size.to_be_bytes())?;
node.setprop_inplace(cstr!("alignment"), &swiotlb_info.align.to_be_bytes())?;
+
+ if let Some(range) = swiotlb_info.fixed_range() {
+ node.appendprop_addrrange(
+ cstr!("reg"),
+ range.start.try_into().unwrap(),
+ range.len().try_into().unwrap(),
+ )?;
+ } else {
+ node.setprop_inplace(cstr!("size"), &swiotlb_info.size.to_be_bytes())?;
+ }
+
Ok(())
}
@@ -540,7 +580,7 @@
num_cpus: usize,
pci_info: PciInfo,
serial_info: SerialInfo,
- swiotlb_info: SwiotlbInfo,
+ pub swiotlb_info: SwiotlbInfo,
}
impl DeviceTreeInfo {
@@ -603,7 +643,7 @@
error!("Failed to read swiotlb info from DT: {e}");
RebootReason::InvalidFdt
})?;
- validate_swiotlb_info(&swiotlb_info)?;
+ validate_swiotlb_info(&swiotlb_info, &memory_range)?;
Ok(DeviceTreeInfo {
kernel_range,
diff --git a/pvmfw/src/memory.rs b/pvmfw/src/memory.rs
index 714815b..a2b7e09 100644
--- a/pvmfw/src/memory.rs
+++ b/pvmfw/src/memory.rs
@@ -21,6 +21,9 @@
use alloc::alloc::alloc_zeroed;
use alloc::alloc::dealloc;
use alloc::alloc::handle_alloc_error;
+use alloc::boxed::Box;
+use buddy_system_allocator::LockedHeap;
+use core::alloc::GlobalAlloc as _;
use core::alloc::Layout;
use core::cmp::max;
use core::cmp::min;
@@ -31,6 +34,7 @@
use core::result;
use hyp::get_hypervisor;
use log::error;
+use once_cell::race::OnceBox;
use spin::mutex::SpinMutex;
use tinyvec::ArrayVec;
@@ -107,6 +111,8 @@
FailedToMap,
/// Error from the interaction with the hypervisor.
Hypervisor(hyp::Error),
+ /// Failure to set `SHARED_POOL`.
+ SharedPoolSetFailure,
}
impl fmt::Display for MemoryTrackerError {
@@ -120,6 +126,7 @@
Self::Overlaps => write!(f, "New region overlaps with tracked regions"),
Self::FailedToMap => write!(f, "Failed to map the new region"),
Self::Hypervisor(e) => e.fmt(f),
+ Self::SharedPoolSetFailure => write!(f, "Failed to set SHARED_POOL"),
}
}
}
@@ -132,6 +139,8 @@
type Result<T> = result::Result<T, MemoryTrackerError>;
+static SHARED_POOL: OnceBox<LockedHeap<32>> = OnceBox::new();
+
impl MemoryTracker {
const CAPACITY: usize = 5;
const MMIO_CAPACITY: usize = 5;
@@ -263,6 +272,31 @@
Ok(())
}
+
+ /// Initialize a separate heap for shared memory allocations.
+ ///
+ /// Some hypervisors such as Gunyah do not support a MemShare API for guest
+ /// to share its memory with host. Instead they allow host to designate part
+ /// of guest memory as "shared" ahead of guest starting its execution. The
+ /// shared memory region is indicated in swiotlb node. On such platforms use
+ /// a separate heap to allocate buffers that can be shared with host.
+ pub fn init_shared_pool(&mut self, range: Range<usize>) -> Result<()> {
+ let size = NonZeroUsize::new(range.len()).unwrap();
+ let range = self.alloc_mut(range.start, size)?;
+ let shared_pool = LockedHeap::<32>::new();
+
+ // SAFETY - `range` should be a valid region of memory as validated by
+ // `validate_swiotlb_info` and not used by any other rust code.
+ unsafe {
+ shared_pool.lock().init(range.start, range.len());
+ }
+
+ SHARED_POOL
+ .set(Box::new(shared_pool))
+ .map_err(|_| MemoryTrackerError::SharedPoolSetFailure)?;
+
+ Ok(())
+ }
}
impl Drop for MemoryTracker {
@@ -305,14 +339,26 @@
Ok(())
}
-/// Allocates a memory range of at least the given size from the global allocator, and shares it
-/// with the host. Returns a pointer to the buffer.
+/// Allocates a memory range of at least the given size that is shared with
+/// host. Returns a pointer to the buffer.
///
/// It will be aligned to the memory sharing granule size supported by the hypervisor.
pub fn alloc_shared(size: usize) -> hyp::Result<NonNull<u8>> {
let layout = shared_buffer_layout(size)?;
let granule = layout.align();
+ if let Some(shared_pool) = SHARED_POOL.get() {
+ // Safe because `shared_buffer_layout` panics if the size is 0, so the
+ // layout must have a non-zero size.
+ let buffer = unsafe { shared_pool.alloc_zeroed(layout) };
+
+ let Some(buffer) = NonNull::new(buffer) else {
+ handle_alloc_error(layout);
+ };
+
+ return Ok(buffer);
+ }
+
// Safe because `shared_buffer_layout` panics if the size is 0, so the layout must have a
// non-zero size.
let buffer = unsafe { alloc_zeroed(layout) };
@@ -341,6 +387,14 @@
let layout = shared_buffer_layout(size)?;
let granule = layout.align();
+ if let Some(shared_pool) = SHARED_POOL.get() {
+ // Safe because the memory was allocated by `alloc_shared` above using
+ // the same allocator, and the layout is the same as was used then.
+ unsafe { shared_pool.dealloc(vaddr.as_ptr(), layout) };
+
+ return Ok(());
+ }
+
let paddr = virt_to_phys(vaddr);
unshare_range(&(paddr..paddr + layout.size()), granule)?;
// Safe because the memory was allocated by `alloc_shared` above using the same allocator, and