Skip to content

~/data

Use the data layer exclusively when fetching data from the API.

WARNING

Make sure to check the tsdocs comments on the functions returned by useDataStore() - this document only provides an abbreviated overview and may be out of date.

GET patterns:

Items by key

  • useDataStore().get_x(key, params?) fetches a single object x.
  • getMany(useDataStore().get_x, ()=> Key[]) fetches a list of objects xs given a dynamic array of keys.

Note: See generated/types.d.ts to understand which field - id, uuid, fid, mbid - serves as key for each resource.

Paginated items

  • useDataStore().get_xs(params) fetches a paginated list of objects xs. Use {} to omit params.
  • getAll(useDataStore().get_xs, params) fetches all objects xs, beginning at the first page and continuing until all pages are fetched. We are using the cursor parameter to paginate through the results, so params in this case cannot contain a cursor field.

Data state machine

All GET actions return a reactive ComputedRef<Data<T>> with the following fields:

  • status: 'notAsked' | 'queued' | 'loading' | 'success' | 'error'
  • data: The API payload (only guaranteed to exist if status === 'success')
  • error: The Error object (only exists if status === 'error')
  • refetch: A function to manually trigger a network request, bypassing the cache TTL.

Always check status or use optional chaining (?.) when accessing data.

Features

  • Cache: Stores fetched data to avoid redundant requests for idempotent verbs. Invalidate entries imperatively or by age.
  • Rate limiter: Limits the frontend to n requests per second (current rate: 20 per second). Individual requests can be prioritized. A deduplication key can mark a new request as superseding (throwing away) old requests, e.g. when a tabbed UI no longer needs to observe them or while a user is typing a search term.
  • DevTools: Observe and manipulate the cache and the configs.

Application

  1. Fetch important data in the script; defer other data in the template

For idempotent verbs, you can request resources directly in the template:

  • If a template node is rendered conditionally, this saves an unnecessary request
  • If a remote object is only needed once, or the result is not important for the first rendering of the page, this simplifies the module state by reducing the scope of the received data object.
  1. Name data according to the API schema

By convention, we name the data object albumResource (resource if it's the only resource on the page) and its optional data field album because the data field has the shape of components['schemas']['Album']. This makes it easy to intuit the shape of a given variable in your script or template. In paginated responses, data.results can be named albums.

Examples

Compute essential data within the script setup (or in a composable):

ts
import { useDataStore } from '~/data'
import { allPages } from '~/data/pagination'

const tracksResource = allPages(useDataStore().get_tracks, {
  ordering: ['disc_number', 'position'],
  album: props.id,
  page_size: paginateBy.value,
  include_channels: true
})

// List of all successfully loaded tracks
const tracks = computed(() => tracksResource.value.data?.results ?? [])

The options parameter allows you to pass headers or cache keys and to refetch when an external signal changes:

ts
const albumResource = dataStore.get_album(props.id, {
  cacheKey: shareToken.value ? `${props.id}-share-${shareToken.value}` : String(props.id),
  headers: shareToken.value ? { 'Link-Share-Token': shareToken.value } : undefined,
  refetchSignal: shareToken
})

Use the headless <Sync> Ui component to limit the scope an idempotent request to a deferred or optional template node:

template
<Sync
  v-if="track.artist_credit?.[0]"
  :use="() => useDataStore().get_artist(track.artist_credit[0]!.artist.id)"
  v-slot="{ value: { data: artist } }"
>
   {{ artist?.name }}

Use a closure (anonymous arrow function) to add options to getMany:

ts
import { getMany, useDataStore } from '~/data'

const linkShareOptions = {
  cacheKey: `${objectId}-share-${link.token}`,
  headers: { 'Link-Share-Token': link.token }
}
const artistsResourceWithOptions = getMany(
  id => useDataStore().get_artist(id, {
    ...linkShareOptions,
    refetchSignal: lastUpdated
  }),
  () => [1, 2, 3]
)
For a detailed up-to-date documentation of the available options, see the source code
ts
import axios, { type CustomParamsSerializer, type ParamsSerializerOptions } from 'axios'
import { defineStore } from 'pinia'
import hash from 'stable-hash'
import type { Split } from 'type-fest'
import { computed, ref, watch, type ComputedRef } from 'vue'

import useLogger from '~/composables/useLogger'
import type { paths } from '~/generated/types'
import { isRateLimiterError, useRateLimiterStore } from './rateLimiter'
import { match, P } from 'ts-pattern'

export { default as DataPagination } from './DataPagination.vue'

const logger = useLogger()

export type Resource<T> = Data<T> & {
  key: string
  refetch: () => void
}
export type CompositeResource<T> = Data<T> & {
  keys: string[]
  refetch: () => void
}

const _names = ['artists', 'albums', 'playlists', 'tracks', 'channels', 'radios/radios', 'tags', 'favorites/tracks', 'history/listenings'] as const satisfies (
  | Exclude<Split<keyof paths, '/'>[3], 'search'>
  | 'radios/radios'
  | 'favorites/tracks'
  | 'history/listenings'
)[]
type Name = (typeof _names)[number]

// Paginated lists
type PathMany<N extends Name> = `/api/v2/${N}/` // Has trailing slash in API
type GetPaginatedResponses<N extends Name> = paths[PathMany<N>]['get']['responses'][200]['content']['application/json']
type Params<N extends Name> = paths[PathMany<N>]['get']['parameters']['query']

// Single items by key
type KeyType<N extends Name> = {
  'artists': 'artists/{id}'
  'albums': 'albums/{id}'
  'playlists': 'playlists/{uuid}'
  'tracks': 'tracks/{id}'
  'channels': 'channels/{composite}'
  'radios/radios': 'radios/radios/{id}'
  'tags': 'tags/{name}'
  'favorites/tracks': 'favorites/tracks/{id}'
  'history/listenings': 'history/listenings/{id}'
}[N]

type PathFirst<N extends Name> = `/api/v2/${KeyType<N>}/` // Has trailing slash in API
type GetFirstResponse<N extends Name> = paths[PathFirst<N>]['get'] extends {
  responses: {
    200: {
      content: {
        'application/json': infer TData
      }
    }
  }
}
  ? TData
  : never

// --------------- Static helpers ----------------

/**
 * Prevents UI flashes when switching parameters (e.g., tabs or pagination)
 * by retaining last known successful data while new resource is pending or loading.
 *
 * Example: const myResourceWithPreviousData = useKeepPreviousData(computed(() => get_albums(my_changing_params)))
 *
 * @param getter A function returning the reactive Data state (e.g., `() => dataStore.get_x(...).value`)
 * @returns computed ref that reflects the resource state with retained data
 */
export const usePreviousData = <T, R extends Resource<T>>(
  getter: () => R
): ComputedRef<R> => {
  const resource = computed(getter)
  const retainedData = ref<T>()

  watch(
    () => resource.value.data,
    (newData) => {
      if (newData !== undefined) {
        retainedData.value = newData as T
      }
    },
    { immediate: true }
  )

  return computed(() =>
    match<Resource<T>>(resource.value)
      .with({ status: 'success' }, () => resource.value)
      .otherwise(() => ({
        ...resource.value,
        data: resource.value.data ?? retainedData.value
      } as const satisfies R)
      )
  )
}

/**
 * Reduces a list of resources into a single resource
 * - Empty list -> Success
 * - Any error taints combined resource as `error`
 * - Successful fetches are appended to `.data` as they come in
 * - If no error is received, status reflects last `queued`/`loading` status
 * -`refetch` will try errored resources first, then successful resources
 * -`key` is the accumulated hash over the first and all subsequent successful resources, i e. it will change with each successful response.
 *
 * @param resources - Array of remote data of arrays of results `Resource<T[]>[]`
 * @returns Single remote data object with array of results `Resource<T[]>`
 */
export const foldResources = <T>(
  resources: Resource<T[]>[]
): Resource<T[]> =>
  match(resources)
    .with([], () => ({
      status: 'success',
      data: [],
      key: 'empty',
      name: 'unknown' as Name,
      requestKey: 'empty',
      lastUpdated: Date.now(),
      refetch: () => { }
    } as const satisfies Resource<T[]>))
    .with([P.any, ...P.array(P.any)], ([head, ...tail]) =>
      tail.reduce<Resource<T[]>>((acc, resource) => match(resource)
        .with({ status: 'error' }, resource => ({
          ...resource, data: acc.data,
          refetch: () => {
            resource.refetch()
            acc.refetch()
          }
        }))
        .with({ status: 'success' }, resource => ({
          ...acc,
          refetch: () => {
            resource.refetch()
            acc.refetch()
          },
          key: acc.key + resource.key,
          data: [...(acc.data ?? []), ...(resource.data ?? [])]
        }))
        .with({ status: P.union('loading', 'queued') }, resource =>
          acc.status !== 'error'
            ? { ...acc, status: resource.status, lastUpdated: resource.lastUpdated }
            : acc
        )
        .otherwise(() => acc),
      { ...head, data: head.data ?? [] }
      )
    )
    .exhaustive()

/**
 * @param item
 * @returns the most unique identifier available per item, assuming every item has at least one key field
 */
export const getKey = (item: { fid: string } | { artist: { fid: string } } | { id: number } | { name: string }) =>
  'fid' in item ? item.fid : 'artist' in item ? item.artist.fid : 'id' in item ? item.id.toString() : item.name

/**
 * Given an array of keys, batch multiple GETs (`get_T(key)`) into a combined `Ref<Data<T[]>>`
 * @see `foldResources` for details
 *
 * @param get_x `id` -> `Ref<Data<T>>`
 * @param getKeys `() => Key[]`
 * @returns Single resource `Ref<Data<T[]>>`
 */
export const getMany = <T>(
  get_x: (id: string | number) => ComputedRef<Resource<T>>,
  getKeys: () => readonly (number | string)[]
) => computed(() =>
  foldResources(getKeys().map((key) => {
    const { value } = get_x(key)
    return { ...value, data: value.data ? [value.data] : [] }
  }))
)

// ======================================================================
// Remote data
type Metadata = {
  name: Name
  key: string
  lastRequested?: number
  deduplicationKey?: GetOptions['deduplicationKey']
  requestKey: string
}
type Data<T> = (
  | { status: 'notAsked', error?: null, data?: T }
  | { status: 'queued', error?: null, data?: T, lastUpdated: number }
  | { status: 'loading', error?: null, data?: T, lastUpdated: number }
  | { status: 'success', error?: null, data: T, lastUpdated: number }
  | { status: 'error', error: Error, data?: T, lastUpdated: number }
) & Metadata

const notAsked = (meta: Metadata) => ({ status: 'notAsked', ...meta }) as const

const setPending = <T>(remoteData: Data<T>): Data<T> & { status: 'queued' } =>
  ({
    ...remoteData,
    status: 'queued',
    error: null,
    lastUpdated: Date.now(),
    lastRequested: Date.now()
  }) as const

const setLoading = <T>(remoteData: Data<T> & { status: 'queued' | 'success' | 'error' }): Data<T> & { status: 'loading' } =>
  ({
    ...remoteData,
    status: 'loading',
    error: null,
    lastUpdated: Date.now()
  }) as const

const setSuccess = <T>(remoteData: Data<T> & { status: 'loading' }, data: T): Data<T> & { status: 'success' } =>
  ({
    ...remoteData,
    status: 'success',
    data,
    error: null,
    lastUpdated: Date.now()
  }) as const

const setError = <T>(remoteData: Data<T> & { status: 'loading' }, error: Error): Data<T> & { status: 'error' } =>
  ({
    ...remoteData,
    status: 'error',
    error,
    lastUpdated: Date.now()
  }) as const

export const invalidate = (remoteData: Data<unknown>) => {
  if ('lastUpdated' in remoteData) remoteData.lastUpdated = 0
}

// ======================================================================
// Search results

type Key = string
type SearchCategory = string

interface GetOptions {
  /**
   * Overrides default cache key. Useful for persisting a single cache slot across varying queries
   * Default: item key (id) for single items or the hash of the parameters (for lists)
   * Override to persist query results in a UI widget when params update so that the displayed results are not empty, e.g. when displaying search results - implementint the stale-while-revalidate pattern
   * Override to purge cached search results that are no longer relevant, e.g. when the user is unlikely to revisit to an earlier set of parameters or an earlier item
   */
  cacheKey?: string
  /**
   * Overrides the rate-limiter queue bucket. Requests with the same key supersede older pending requests
   * Default: resource name and set of query parameters
   * Override for quick live UI, e.g. typing a query or changing tabs - will debounce intermediate filters
   */
  deduplicationKey?: string
  /**
   * Custom HTTP headers to merge into the request (e.g., `Link-Share-Token`).
   */
  headers?: Record<string, string>
  /**
   * Reactive signal (ref, computed, or getter) that triggers a background refetch when changed
   * Use for external events (e.g., `() => store.state.moderation.lastUpdate`)
   */
  refetchSignal?: Parameters<typeof watch>[0]
}

export const useDataStore = defineStore('data', () => {
  const rateLimiter = useRateLimiterStore()

  const config = ref<{
    maxAge: number
    paramsSerializer: ParamsSerializerOptions | CustomParamsSerializer
  }>({
    maxAge: 5 * 60000,
    paramsSerializer: { indexes: null }
  })

  // ------------- Composable Factory --------------- //

  /**
   * State machine and rate limiter for single items and lists
   *
   * @param cacheMap
   * @param key
   * @param meta
   * @param fetcher
   * @param options
   */
  const useGet = <T>(
    cacheMap: Map<string, Data<T>>,
    key: string,
    meta: { name: Name, deduplicationKey: string, requestKey: string },
    fetcher: () => Promise<T>,
    options: Pick<GetOptions, 'refetchSignal'> = {}
  ) => {
    const cached = computed<Data<T>>({
      get: () => cacheMap.get(key) ?? notAsked({ ...meta, key }),
      set: value => cacheMap.set(key, value)
    })

    const scheduleFetch = (rateLimiterConfig?: Parameters<typeof rateLimiter.greenlight>[1]) => async () => {
      try {
        cached.value = setPending({ ...cached.value, requestKey: meta.requestKey })
        await rateLimiter.greenlight<SearchCategory>(meta.deduplicationKey, rateLimiterConfig)
        cached.value = setLoading(cached.value as Data<T> & { status: 'queued' })

        const data = await fetcher()

        cached.value = match(cached.value)
          .with({ status: 'loading' }, state => setSuccess(state, data))
          .otherwise(state => state)
      } catch (error) {
        if (isRateLimiterError(error as Error)) {
          logger.info(error)
          // Revert superseded task to allow future requests
          cached.value = notAsked({ ...meta, key })
        } else {
          logger.error(`Error fetching ${meta.name} [${key}]:`, error)
          cached.value = match(cached.value)
            .with({ status: 'loading' }, state => setError(state, error as Error))
            .otherwise(state => state)
        }
      }
    }

    if (
      cached.value.status === 'notAsked'
      || cached.value.lastUpdated < Date.now() - config.value.maxAge
      || cached.value.requestKey !== meta.requestKey
    ) {
      scheduleFetch()()
    }

    if (options.refetchSignal) {
      watch(options.refetchSignal, scheduleFetch())
    }

    return computed(() => ({
      ...(cacheMap.get(key)!),
      key,
      refetch: scheduleFetch({ priority: true })
    }))
  }

  // ------------- Caches --------------- //

  const single = ref({
    'artists': new Map<Key, Data<GetFirstResponse<'artists'>>>(),
    'albums': new Map<Key, Data<GetFirstResponse<'albums'>>>(),
    'playlists': new Map<Key, Data<GetFirstResponse<'playlists'>>>(),
    'tracks': new Map<Key, Data<GetFirstResponse<'tracks'>>>(),
    'channels': new Map<Key, Data<GetFirstResponse<'channels'>>>(),
    'radios/radios': new Map<Key, Data<GetFirstResponse<'radios/radios'>>>(),
    'tags': new Map<Key, Data<GetFirstResponse<'tags'>>>(),
    'favorites/tracks': new Map<Key, Data<GetFirstResponse<'favorites/tracks'>>>(),
    'history/listenings': new Map<Key, Data<GetFirstResponse<'history/listenings'>>>()
  } as const)

  const paginated = ref({
    'artists': new Map<Key, Data<GetPaginatedResponses<'artists'>>>(),
    'albums': new Map<Key, Data<GetPaginatedResponses<'albums'>>>(),
    'playlists': new Map<Key, Data<GetPaginatedResponses<'playlists'>>>(),
    'tracks': new Map<Key, Data<GetPaginatedResponses<'tracks'>>>(),
    'channels': new Map<Key, Data<GetPaginatedResponses<'channels'>>>(),
    'radios/radios': new Map<Key, Data<GetPaginatedResponses<'radios/radios'>>>(),
    'tags': new Map<Key, Data<GetPaginatedResponses<'tags'>>>(),
    'favorites/tracks': new Map<Key, Data<GetPaginatedResponses<'favorites/tracks'>>>(),
    'history/listenings': new Map<Key, Data<GetPaginatedResponses<'history/listenings'>>>()
  } as const)

  // ------------- Resources --------------- //

  const createItemResource = <N extends Name>(name: N) =>
    (id: string | number,
      options: Pick<GetOptions, 'refetchSignal' | 'cacheKey' | 'headers'> = {}
    ) => useGet(
      single.value[name] as Map<Key, Data<GetFirstResponse<N>>>,
      options.cacheKey ?? String(id),
      { name, deduplicationKey: `${name}/${String(id)}`, requestKey: options.cacheKey ?? String(id) },
      async () => {
        const { data } = await axios.get<GetFirstResponse<N>>(`${name}/${id}/`, {
          headers: options.headers
        })
        return data
      },
      options
    )

  const createListResource = <N extends Name>(name: N) =>
    (params: Params<N> = {},
      options: Pick<GetOptions, 'refetchSignal' | 'deduplicationKey' | 'cacheKey'> = {}
    ) => useGet(
      paginated.value[name] as Map<Key, Data<GetPaginatedResponses<N>>>,
      options.cacheKey ?? hash(params),
      { name, deduplicationKey: options.deduplicationKey ?? hash([name, params]), requestKey: hash(params) },
      async () => {
        const { data } = await axios.get<GetPaginatedResponses<N>>(name, { params, paramsSerializer: config.value.paramsSerializer })
        return data
      },
      options
    )

  return {
    config,
    'items': single,
    'searches': paginated,

    // Single items
    'get_artist': createItemResource('artists'),
    'get_album': createItemResource('albums'),
    'get_track': createItemResource('tracks'),
    'get_channel': createItemResource('channels'),
    'get_radio': createItemResource('radios/radios'),
    'get_tag': createItemResource('tags'),
    'get_playlist': createItemResource('playlists'),
    'get_favorites/track': createItemResource('favorites/tracks'),
    'get_history/listening': createItemResource('history/listenings'),

    // Paginated lists
    'get_artists': createListResource('artists'),
    'get_albums': createListResource('albums'),
    'get_tracks': createListResource('tracks'),
    'get_channels': createListResource('channels'),
    'get_radios': createListResource('radios/radios'),
    'get_tags': createListResource('tags'),
    'get_playlists': createListResource('playlists'),
    'get_favorites/tracks': createListResource('favorites/tracks'),
    'get_history/listenings': createListResource('history/listenings'),

    // Resource observers (global)
    'cachedResources': computed(() => [
      ...Object.values(paginated.value).flatMap(entries => [...entries.values()]),
      ...Object.values(single.value).flatMap(entries => [...entries.values()])
    ]),

    'latestResourcesByDeduplicationKey': (predicate: (deduplicationKey: string) => boolean) => computed(() =>
      [
        ...Object.values(paginated.value).flatMap(entries => [...entries.values()]),
        ...Object.values(single.value).flatMap(entries => [...entries.values()])
      ].filter((resource, _, allResources) => {
        if (typeof resource.deduplicationKey !== 'string') return false
        if (!predicate(resource.deduplicationKey)) return false
        const newerResourceWithSameKey = allResources.find(r =>
          r.deduplicationKey === resource.deduplicationKey
          && (r.lastRequested ?? 0) > (resource.lastRequested ?? 0)
        )
        return !newerResourceWithSameKey
      })
    ),

    'numberOfCachedResources': computed(() =>
      Object.values(paginated.value).reduce((sum, entries) => sum + entries.size, 0)
      + Object.values(single.value).reduce((sum, entries) => sum + entries.size, 0)
    ),

    'numberOfQueuedResources': computed(() =>
      Object.values(paginated.value).reduce((sum, entries) => sum + [...entries].filter(([_, v]) => v.status === 'queued').length, 0)
      + Object.values(single.value).reduce((sum, entries) => sum + [...entries].filter(([_, v]) => v.status === 'queued').length, 0)
    ),

    'numberOfLoadingResources': computed(() =>
      Object.values(paginated.value).reduce((sum, entries) => sum + [...entries].filter(([_, v]) => v.status === 'loading').length, 0)
      + Object.values(single.value).reduce((sum, entries) => sum + [...entries].filter(([_, v]) => v.status === 'loading').length, 0)
    )
  }
})

Scope and roadmap

The guiding principle of the ~/data feature is to stay close to the funkwhale API and avoid any abstraction or generalization that is not immediately useful for the app.

We are planning to cover more verbs in the future. For now, stick to out-of-band imperative axios calls for anything other than GET.

MethodSafe (readonly)IdempotentRequest bodyResponse bodyInvalidates cache
GETYesYesNoYesNo
HEADYesYesNoNoNo
PUTNoYesYesOptionalYes
DELETENoYesOptionalOptionalYes
POSTNoNoYesYesYes
QUERYYesYesYesYesNo
PATCHNoNoYesYesYes
OPTIONSYesYesOptionalYesNo
TRACEYesYesNoYesNo