> For the complete documentation index, see [llms.txt](https://developerdocs.instructure.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developerdocs.instructure.com/services/dap/dap-cli-readme/dap-cli-reference/dap-cli-reference-incremental.md). # dap incremental Download changes from a table within a specified time range. Generates and downloads only the new or modified records of a table within a specified timeframe. This efficient, resource-effective approach helps keep datasets up-to-date with minimal overhead. Incremental queries enable near-real-time updates, reducing the need for frequent full dataset downloads. ### Usage ``` dap [arguments] incremental [flags] ``` ### Options **`--namespace `**\ Specifies the data source (namespace). Available options: {canvas, canvas\_logs, catalog, new\_quizzes}. **`--table `**\ Specifies the table(s) to fetch data from. Can be a single table name, a comma separated list of table names or the special `all` keyword to fetch all tables in the namespace. **`--since `**\ Start timestamp for the incremental query. Examples: `2024-12-01T09:30:00Z` **`--until `**\ End timestamp for the incremental query. Examples: `2024-12-01T09:30:00Z` **`--format (default: JSONL)`**\ Defines the output format. Available options: {CSV, JSONL, Parquet, TSV}. **`--output-directory (default: downloads)`**\ Specifies the absolute or relative path to the output directory where the result will be stored. ### Global options **`--client-id `**\ Client ID obtained from the Identity Service. Skip, if `DAP_CLIENT_ID` environment variable is set. **`--client-secret `**\ Client Secret obtained from the Identity Service. Skip, if `DAP_CLIENT_SECRET` environment variable is set. **`-h, --help`**\ Displays help information for the command. ### Output The output structure is identical to [`dap snapshot`](/services/dap/dap-cli-readme/dap-cli-reference/dap-cli-reference-snapshot.md): gzipped data files inside a `job_/` subdirectory. Each decompressed JSONL line contains `key`, `value`, and `meta` fields. In incremental results, `meta.action` may be present: * `"U"` — the record was inserted or updated (upsert). * `"D"` — the record was hard-deleted. When `action` is `"D"`, `value` may be absent. If there are no changes in the requested time range, the command exits with code `0` and the output file will be empty. ### Error handling | Scenario | Exit code | Error output | | ------------------------------------------ | --------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `--since` precedes earliest available data | 5 | `ERROR - Operation incremental failed. Reason: The 'since' parameter in the request (...) precedes the earliest data available (...).` | | Table was recently reloaded | 5 | Same as above, with a message advising to drop the table and start with a fresh snapshot. | | Invalid table name | 5 | `ERROR - Operation incremental failed. Reason: The table name '' specified in the request is not valid.` | | Missing required flag | 2 | `Error: Missing option '--since'.` | {% hint style="info" %} The `--since` timestamp must include a timezone designator (e.g. `Z` for UTC or `+02:00`). Timestamps without a timezone are rejected. {% endhint %} ### Examples Get new or modified records of the `courses` table from the `canvas` namespace since November 1, 2024:\ `$ dap incremental --namespace canvas --table courses --since 2024-11-01T00:00:00Z` Get new or modified records of all tables from the `canvas` namespace between November 1, 2024, and November 30, 2024:\ `$ dap incremental --namespace canvas --table all --since 2024-11-01T00:00:00Z --until 2024-11-30T00:00:00Z` Scripting-friendly example with suppressed logs:\ `$ dap --non-interactive --loglevel warning incremental --namespace canvas --table accounts --since 2024-11-01T00:00:00Z --output-directory ./data` ### Related Resources
Key ConceptsGet familiar with the key concepts in DAP.Key Concepts
Rate Limits & PoliciesLearn more about the limits and our policies in DAP.Rate Limits & Policies
DatasetsDiscover the available namespaces and tables.Datasets
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://developerdocs.instructure.com/services/dap/dap-cli-readme/dap-cli-reference/dap-cli-reference-incremental.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.