Merge "Add documentation on pkvm_hypervisor_events view"
diff --git a/docs/debug/tracing.md b/docs/debug/tracing.md
index ebc0ac3..facd9d0 100644
--- a/docs/debug/tracing.md
+++ b/docs/debug/tracing.md
@@ -16,11 +16,13 @@
 * Only boot clock is supported, and there is no way for user space to change the tracing_clock.
 * Hypervisor tracing periodically polls the data from the hypervisor, this is different from the
   regular ftrace instance which pushes the events into the ring buffer.
+* Resetting ring buffers (by clearing the trace file) is only supported when there are no active
+  readers. If the trace file is cleared while there are active readers, then the ring buffers will
+  be cleared after the last reader disconnects.
+* Changing the size of the ring buffer while the tracing session is active is also not supported.
 
 Note: the list above is not exhaustive.
 
-TODO(b/271412868): add more documentation on the user space interface.
-
 ### Perfetto integration
 
 [Perfetto](https://perfetto.dev/docs/) is an open-source stack for performance instrumentation and
@@ -87,6 +89,61 @@
 tracebox -t 15s -b 32mb hyp
 ```
 
+### Analysing traces using SQL
+
+On top of visualisation, Perfetto also provides a SQL interface to analyse traces. More
+documentation is available at https://perfetto.dev/docs/quickstart/trace-analysis and
+https://perfetto.dev/docs/analysis/trace-processor.
+
+Hypervisor events can be queried via `pkvm_hypervisor_events` SQL view. You can load that view by
+calling `SELECT IMPORT("pkvm.hypervisor");`, e.g.:
+
+```sql
+SELECT IMPORT("pkvm.hypervisor");
+SELECT * FROM pkvm_hypervisor_events limit 5;
+```
+
+Below are some SQL queries that might be useful when analysing hypervisor traces.
+
+**What is the longest time CPU spent in hypervisor, grouped by the reason to enter hypervisor**
+```sql
+SELECT IMPORT("pkvm.hypervisor");
+
+SELECT
+  cpu,
+  reason,
+  ts,
+  dur
+FROM pkvm_hypervisor_events
+JOIN (
+  SELECT
+    MAX(dur) as dur2,
+    cpu as cpu2,
+    reason as reason2
+  FROM pkvm_hypervisor_events
+  GROUP BY 2, 3) AS sc
+ON
+  cpu = sc.cpu2
+  AND dur = sc.dur2
+  AND (reason = sc.reason2 OR (reason IS NULL AND sc.reason2 IS NULL))
+ORDER BY dur desc;
+```
+
+**What are the 10 longest times CPU spent in hypervisor because of host_mem_abort**
+```sql
+SELECT
+  hyp.dur as dur,
+  hyp.ts as ts,
+  EXTRACT_ARG(slices.arg_set_id, 'esr') as esr,
+  EXTRACT_ARG(slices.arg_set_id, 'addr') as addr
+FROM pkvm_hypervisor_events as hyp
+JOIN slices
+ON hyp.slice_id = slices.id
+WHERE hyp.reason = 'host_mem_abort'
+ORDER BY dur desc
+LIMIT 10;
+```
+
 ## Microdroid VM tracing
 
 IMPORTANT: Tracing is only supported for debuggable Microdroid VMs.