Skip to content

Data streaming

Some of the API endpoints support streaming all data available as a line-break delimited list of documents.

This will offer you the possibility to retrieve all the data about a particular device or folder without having to chain paginations.

To use the streaming variant of an endpoint, simply append /stream to its path, like this:

GET /api/v1/devices/my-cool-device/readings/stream

The API will then stream all the available data to your HTTP client. You should only use the stream API with an HTTP client that can process a potentially huge amount of data in a memory-efficient fashion. In most languages, this concept will be called streaming.

The response won't be a valid JSON object; instead, there will be one JSON object per line. This allows a client to split binary chunks by \n and mapping over the stream of lines with any JSON decoder, instead of having to wait until the whole document is transmitted and then decoding everything at once.

Normal responseStreaming response
{
  "body": [
    {"id": 1}, 
    {"id": 2}, 
    {"id": 3}, 
    {"id": 4}
  ]
}

{"id": 1}
{"id": 2}
{"id": 3}
{"id": 4}

Supported endpoints

  • Device readings list
  • Folder readings list
  • Device packets list
  • Folder packets list

All query parameters from the listed endpoints are also supported when streaming, except the pagination params retrieve_after and limit.

In the device endpoints, selectors will also work.

Further reading

HTTP Streaming in Elixir describes why and how to process huge HTTP responses with streaming instead of waiting for the whole response to be transmitted.