Fix summary loader lifecycle

Previous CL accidentally cleared the listening bit too early making
it never actually tell the SummaryProviders to stop listening.

Change-Id: I6af1acc861f6987b33641c40637354c75cf4d708
Fixes: 28306872
1 file changed