# Caching Configure caching in gpui-query: how NoCache, Ttl, and StaleWhileRevalidate policies decide when data is fresh, stale, or refetched in the background. Every query in gpui-query has a cache policy. The policy determines whether a fetch happens at all, and whether stale data gets served while a refetch runs in the background. Picking the right policy is the main lever you have for tuning how "live" your data feels versus how many network requests your app makes. ## Cache policies There are three variants of `CachePolicy`. The default is `Ttl { ttl_ms: 60_000 }`, a 60-second TTL. You can override it per query or set a different default when constructing the [QueryClient](/docs/api/query-client). ### NoCache ```rust CachePolicy::NoCache ``` Every access triggers a fresh fetch. No cached data is stored or returned. Use this for data that changes constantly and where showing a stale value is worse than showing nothing. The practical effect: `begin_request` always returns `Started`, never `CacheHit`. ### Ttl ```rust CachePolicy::Ttl { ttl_ms: 30_000 } ``` Data is fresh for `ttl_ms` milliseconds after it was last fetched. While fresh, accessing the query returns `CacheHit` and skips the network entirely. After the TTL expires, the next access starts a new fetch. This is the default policy. The built-in default TTL is 60 seconds (60,000 ms). For most read-heavy UIs this is a reasonable starting point. ### StaleWhileRevalidate ```rust CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 } ``` This policy splits the cache lifetime into two windows. **Fresh window** (0 to `ttl_ms`): data is served from cache. No fetch happens. Same as `Ttl`. **Stale window** (`ttl_ms` to `ttl_ms + stale_ms`): cached data is returned immediately, and a background refetch starts. The caller gets `StaleCacheHit` with a `request_id` for the background fetch. **Expired** (past `ttl_ms + stale_ms`): the data is too old to serve. A normal fetch happens, same as a cache miss. Use this when your UI benefits from showing slightly outdated data rather than flashing a loading spinner. Feed timelines, dashboards, and list views are common candidates. ## How freshness is decided `CachePolicy` exposes several methods that control the caching decision internally. You normally do not call these yourself (the library does), but understanding them helps reason about behavior. - **`is_fresh(age_ms)`** - returns `true` when `age_ms` is within the TTL window - **`is_stale_but_serveable(age_ms)`** - returns `true` when the age is past TTL but within the stale window (only `StaleWhileRevalidate`) - **`is_expired(age_ms)`** - returns `true` when the age exceeds the total valid window. `NoCache` always returns `true` - **`can_short_circuit(age_ms)`** - whether the policy can skip a fetch entirely (true for `Ttl` and `StaleWhileRevalidate` when data is fresh) - **`can_serve_stale(age_ms)`** - whether stale data can be returned (true only for `StaleWhileRevalidate`) - **`total_valid_ms()`** - returns `ttl_ms + stale_ms` for SWR, `ttl_ms` for `Ttl`, `None` for `NoCache` ## Setting the policy Pass the policy when creating a `QueryClient` to set the app-wide default. ```rust cx.set_global(QueryClient::new( CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 }, RequestPolicy::LatestWins, )); ``` Override per query through `QueryOptions`: ```rust let opts = QueryOptions::new(QueryKey::from(["notifications"])) .cache_policy(CachePolicy::NoCache); ``` Or use `resource_with_policies` on the client directly: ```rust let entity = client.resource_with_policies::, MyError>( QueryKey::from(["notifications"]), CachePolicy::NoCache, RequestPolicy::LatestWins, cx, ); ``` ## Request deduplication When multiple parts of your app request the same key at the same time, the `RequestPolicy` decides what happens. ### LatestWins (default) A new request cancels the in-flight one and starts fresh. This is the right choice when you always want the most recent data and do not care about wasting the earlier request. ```rust RequestPolicy::LatestWins ``` ### IgnoreWhileLoading If a request is already in progress, new requests for the same key are ignored. The original request runs to completion. Use this when fetches are idempotent and you want to avoid redundant work. ```rust RequestPolicy::IgnoreWhileLoading ``` ## What happened: QueryBeginResult When you call `begin_request` on a resource, the return value tells you exactly what happened. ```rust pub enum QueryBeginResult { Started { request_id, status, replaced_request_id }, CacheHit, StaleCacheHit { request_id, status, replaced_request_id }, IgnoredWhileLoading { active_request_id }, } ``` - **`Started`** - a new fetch was kicked off. `replaced_request_id` is `Some` if this request cancelled a previous one (under `LatestWins`). - **`CacheHit`** - the cache is fresh, no fetch needed. Use the cached data. - **`StaleCacheHit`** - data is stale but still serveable. Return the stale data to the UI immediately, then use the `request_id` to run a background fetch. - **`IgnoredWhileLoading`** - a request was already running and the policy is `IgnoreWhileLoading`. ## Forcing a refetch `QueryFetchMode` controls whether the cache is consulted. - **`Normal`** (default) - respects the cache policy. Returns `CacheHit` if fresh. - **`Force`** - bypasses cache freshness. Always starts a new fetch. Still respects `IgnoreWhileLoading` if a request is already in flight. Use `Force` when the user explicitly requests fresh data (pull-to-refresh, a manual reload button). ```rust client.force_fetch_query::( QueryKey::from(["users", "42"]), cache_policy, request_policy, now_ms, cx, ); ``` ## Bulk operations on QueryClient The [QueryClient](/docs/api/query-client) has bulk methods that operate across all resources matching a filter. These use `QueryKeyFilter` to select which queries to touch. ### Invalidation `invalidate_queries` clears cache freshness on matching resources. The next access to those queries will trigger a refetch. It does not cancel in-flight requests. ```rust let client = cx.global::(); client.invalidate_queries( &QueryKeyFilter::Prefix(&QueryKey::from(["users"])), cx, ); ``` This is the standard pattern after a mutation: invalidate the affected query prefix so the next read fetches fresh data. ### Removal `remove_queries` drops matching resources from the registry entirely. Unlike invalidation, which keeps the entity and just expires its cache, removal deletes the entity. ```rust client.remove_queries(&QueryKeyFilter::All, cx); ``` ### QueryKeyFilter variants - **`Exact(&key)`** - matches only the exact key - **`Prefix(&key)`** - matches all keys that start with the given prefix segments - **`All`** - matches every key Prefix matching is hierarchical. `Prefix(&QueryKey::from(["users"]))` matches `["users"]`, `["users", "42"]`, and `["users", "42", "posts"]`. ## Optimistic updates Sometimes you want to update the UI before the server confirms the change. `set_query_data` writes a value directly into the cache, storing the previous value for rollback. ```rust let client = cx.global::(); client.set_query_data::, MyError>( &key, updated_user_list, cx, ); ``` If the mutation later fails, roll back to the previous data: ```rust client.rollback_query_data::, MyError>(&key, cx); ``` This restores the data that was in the resource before `set_query_data` was called. If there was no previous data, the rollback does nothing and returns `false`. ## Garbage collection Queries that no component is observing will linger in memory until garbage collection picks them up. The `gc` method on [QueryClient](/docs/api/query-client) removes resources that have been idle longer than `gc_time_ms`. ```rust let client = cx.global::(); client.gc(cx, now_ms); ``` The default GC time is 5 minutes (300,000 ms). You can change this per query through `QueryOptions`: ```rust let opts = QueryOptions::new(key).gc_time_ms(10 * 60 * 1_000); // 10 minutes ``` Or change the client-wide default during construction: ```rust cx.set_global( QueryClient::new(cache_policy, request_policy) .with_gc_time(10 * 60 * 1_000), ); ``` ## Server wins with `Fetched` A fetcher normally returns `Result`, and the resource uses whatever `CachePolicy` the caller supplied through `QueryOptions`. That policy is decided entirely on the client. Sometimes the server knows better. A `Cache-Control: max-age=30` response header is more authoritative than a TTL you guessed at compile time. The `*_with_policy` hooks let a fetcher return `Result, E>` instead, carrying a server-derived `CachePolicy` that overrides the caller's per-query policy once the fetch resolves successfully. - **`use_query_with_policy(options, fetcher, cx)`** - same shape as `use_query` (options-first, signal-accepting fetcher, returns `(Entity>, Subscription)`), but the fetcher returns `Result, E>`. - **`fetch_query_with_policy(&entity, fetcher, cx)`** - same shape as `fetch_query`, but the fetcher returns `Result, E>`. `Fetched` has three constructors: ```rust // No override: the resource keeps the caller's policy. Fetched::new(data) // Override: the resource's policy is replaced with the server's. Fetched::with_policy(data, policy: CachePolicy) // (`persist` feature) Attach opaque metadata that flows into PersistedEntry::meta. Fetched::with_meta(self, meta) ``` A fetcher that just parsed the response headers can push that TTL back into the resource: ```rust use gpui_query::{ use_query_with_policy, CachePolicy, Fetched, QueryKey, QueryOptions, }; use gpui_query_http::cache_policy_from_headers; let (entity, _sub) = use_query_with_policy( QueryOptions::new(QueryKey::from(["notifications"])), |_signal| async move { let resp = client.get(url).send().await?; let policy = cache_policy_from_headers(resp.headers()) .unwrap_or(CachePolicy::NoCache); Ok(Fetched::with_policy(resp.json().await?, policy)) }, cx, ); ``` The server policy replaces the resource's stored policy immediately after `complete_success`, so subsequent freshness and SWR checks use the server's TTL. Returning `Fetched::new(data)` (or using the plain `Result` hooks) leaves the caller's policy untouched. Server-wins is strictly opt-in. `Fetched::with_meta` requires the `persist` feature. The opaque `meta` flows into `PersistedEntry::meta` when the resource is persisted, so it survives a cold start. See [Persistence](/docs/guides/persistence). For the full HTTP `Cache-Control` and ETag flow, see [HTTP caching](/docs/guides/http-caching). ## Putting it together A typical setup looks like this: ```rust // App-wide defaults: 30s TTL with 60s stale window, latest-wins dedup, 10-minute GC cx.set_global( QueryClient::new( CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000, stale_ms: 60_000 }, RequestPolicy::LatestWins, ) .with_gc_time(10 * 60 * 1_000), ); ``` Then override per query as needed. A real-time notifications feed might use `NoCache`. A rarely-changing config endpoint might use `Ttl { ttl_ms: 300_000 }`. Most things fall somewhere in between. See [Queries](/docs/api/queries) for how to define queries that use these policies, [QueryClient](/docs/api/query-client) for the full client API, and [Error Handling](/docs/guides/error-handling) for what happens when a fetch fails.