Getting Started

A step-by-step guide for IT admins, analysts, and non-technical users.

1. Get API Client ID and Secret

To access the DAP API, you will need a Client ID and Secret. These are generated via the Instructure Identity Services.

Steps:

1. Log into Instructure Identity Services.

2. Select your institution from the drop-down menu and log in.

3. Once authorized, navigate to the dashboard and click Add New Key.

4. Enter a name for the key and set the expiration time.

5. Copy the Client ID and Secret when they appear. Note: These are displayed only once. If you lose them, you will need to generate new ones.

2. Install DAP CLI on Your Computer

The DAP CLI tool allows you to interact with the Canvas Data 2 API. Installation steps differ slightly depending on your operating system.

For Mac:

1. Install Xcode Developer Tools:

   xcode-select --install

2. download and install Python 3.10+ from here:

   python3 --version

3. Install PIP (if not installed by default):

   pip3 --version

4. Install the DAP CLI tool with PostgreSQL support:

   pip3 install "instructure-dap-client[postgresql]"

For Windows:

1. Install Python 3.10+ from here.

2. Install the DAP CLI tool using the Windows command prompt:

   pip3 install "instructure-dap-client[postgresql]"

If you miss installing an extra feature, the library will not be able to synchronize data with a database, and you may get an error message similar to the following:

```
ERROR - missing dependency: `asyncpg`; you may need to run `pip install pysqlsync[postgresql]`
```

3. Store Client Credentials in Environment Variables

For secure access to the API, it's recommended to store your credentials as environment variables. This prevents sensitive information from being exposed in command-line arguments.

MacOS/Linux:

1. Open Terminal and run the following commands, replacing placeholders with your actual Client ID and Secret:

   export DAP_API_URL='https://api-gateway.instructure.com'
   export DAP_CLIENT_ID='your_canvas_data_client_id'
   export DAP_CLIENT_SECRET='your_canvas_data_secret'

2. Restart Terminal for changes to take effect.

Windows:

Follow this guide to setting environment variables or use the set command in the Windows command line:

set DAP_API_URL=https://api-gateway.instructure.com
set DAP_CLIENT_ID=your_canvas_data_client_id
set DAP_CLIENT_SECRET=your_canvas_data_secret

Unless you set environment variables, you need to pass endpoint URL, Client ID, and Secret to the dap command explicitly:

dap --base-url https://api-gateway.instructure.com --client-id=us-east-1#0c59cade-...-2ac120002 --client-secret=xdEC0lI...4X4QBOhM incremental --namespace canvas --table accounts --since 2022-07-13T09:30:00+02:00

4. Work with Databases

DAP CLI allows you to interact with PostgreSQL or MySQL databases. You will need the connection string of your database for DAP to function correctly.

Connection String Format:

protocol://username:password@host:port/database_name

Example for PostgreSQL:

postgresql://user:password@localhost:5432/mydatabase

Store Connection String in Environment Variables

MacOS/Linux:

1. Open Terminal and run the following commands, replacing placeholders with your actual Client ID and Secret:

   export DAP_CONNECTION_STRING=postgresql://user:password@localhost:5432/mydatabase

2. Restart Terminal for changes to take effect.

Windows:

Follow this guide to setting environment variables or use the set command in the Windows command line:

set DAP_CONNECTION_STRING=postgresql://user:password@localhost:5432/mydatabase

Obtain a Full Snapshot of a Table

Use the dap initdb command to download a full snapshot of a table and store it in your database.

Command:

dap initdb --connection-string postgresql://user:password@localhost/mydb --namespace canvas --table accounts

Regular use of snapshots is not recommended, as they are resource-intensive for the API and costly to process on the client side.

Synchronize Data with a Table

After obtaining a snapshot, keep your database updated with the dap syncdb command. This ensures incremental changes are applied to your table.

Command:

dap syncdb --connection-string postgresql://user:password@localhost/mydb --namespace canvas --table accounts

Changing the Temporary Storage Location

If you need to change the temporary storage directory for data processing, you can configure the location using the following command-line option:

dap syncdb --temp-dir /new/temp/location --connection-string postgresql://user:password@localhost/mydb --namespace canvas --table accounts

5. Export Data

You can export data using either the snapshot or incremental methods, depending on your use case.

Snapshot Export

Use dap snapshot command to download a full copy of a table at a point in time.

Command:

dap snapshot --namespace canvas --table accounts

Regular use of snapshots is not recommended, as they are resource-intensive for the API and costly to process on the client side.

Incremental Export

The dap incremental commands captures only the data that has changed since your last export.

Command:

dap incremental --namespace canvas --table accounts --since YYYY-MM-DDTHH:MM:SSZ