Using the Stargate Document API

Stargate is a data gateway deployed between client applications and a database. The Document API plugin that exposes CRUD access to data stored as unstructured JSON documents in collections.

If the Document API is used with Apache Cassandra, the document indexing is accomplished with secondary indexes. If the Document API is used with DataStax Enterprise, SAI indexing is used. The blog The Stargate Cassandra Documents API describes the underlying structure used to store collections.

Prerequisites

To use Stargate you need:

Docker installed and running, if using Docker
cURL to run REST, Document, or GraphQL queries

If you are looking to just get started, DataStax Astra Database-as-a-Service can get you started with no install steps.

This image contains the Cassandra Query Language (CQL), REST, Document, GraphQL APIs, and GraphQL Playground, along with an Apache Cassandra^™ 3.11 backend.

docker pull stargateio/stargate-3_11:v1.0.41

Start the Stargate container in developer mode. Developer mode removes the need to set up a separate Cassandra instance and is meant for development and testing only.

docker run --name stargate \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 8082:8082 \
  -p 127.0.0.1:9042:9042 \
  -d \
  -e CLUSTER_NAME=stargate \
  -e CLUSTER_VERSION=3.11 \
  -e DEVELOPER_MODE=true \
  stargateio/stargate-3_11:v1.0.41

Swagger UI for the Document API

Once you have started the docker container, you can access the Document API in a browser at localhost:8082/swagger-ui. Adding parameter information, you can generate cURL commands to execute and display results that will return.

Using the Auth API to generate an auth token

In order to use the Stargate Document API, an authorization token must be generated to access the interface.

The step below uses cURL to access the REST interface to generate the needed token.

Generate an auth token

First generate an auth token that is required in each subsequent request in the X-Cassandra-Token header. Note the port for the auth service is 8081.

curl -L -X POST 'http://localhost:8081/v1/auth' \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "username": "cassandra",
    "password": "cassandra"
}'

You should receive a token in the response.

{"authToken":"{auth-token}"}

Use the auth token

Store the auth token in an environment variable to make it easy to use with cURL.

export AUTH_TOKEN={auth-token}

Using Postman

If you prefer, you can use Postman as a client interface for exploring the Document API (download here). See Tooling Resources for links to downloadable collections and environments. Instructions for configuring the environments for Stargate OSS and Astra are available. You can immediately import the documentation code examples and explore!

Now you’re ready to use the Document API for CRUD operations.

Creating or dropping a namespace

In order to use the Document API, you must create the namespace as a container that will store collections, which in turn store documents. Documents can themselves hold multiple documents. Multiple collections are contained in a namespace, but a collection cannot be contained in multiple namespaces.

Only namespaces need to be specifically created. Collections are specified when a document is inserted. An optional setting, replicas, defines the number of data replicas the database will store for the namespace. If no replica is defined, then for a namespace in a single datacenter cluster, the default is 1, and for a multiple-datacenter cluster, the default is 3 for each datacenter.

Creating a namespace

Simple namespace

Send a POST request to /v2/schemas/namespaces. In this example we use myworld for the name, and no replicas setting, to default to 1.

cURL command (/v2)
Result

curl --location --request POST 'localhost:8082/v2/schemas/namespaces' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "name": "myworld"
}'

{"name":"myworld"}

The authorization token and the content type are passed with --header. The token must be identified as X-Cassandra-Token so that cluster recognizes the token and its value. The specified name for the namespace is passed as JSON data using --data. For shorthand, cURL can use -L for --location, -X for --request, -H for --header, and -d for --data.

Set replicas in simple namespace

To set the replicas, send a POST request to /v2/schemas/namespaces. In this example we use myworld for the name, and 2 for the number of data replicas.

cURL command (/v2)
Result

curl -L -X POST 'localhost:8082/v2/schemas/namespaces' \
-H "X-Cassandra-Token: $AUTH_TOKEN" \
-H 'Content-Type: application/json' \
-d '{
    "name": "myworld",
    "replicas": 1
}'

{"name":"myworld"}

Namespace for multiple datacenters

For a multiple-datacenter cluster, a namespace is defined datacenters. Send a POST request to /v2/schemas/namespaces. In this example we use myworld-dcs for the name, the datacenters are dc1 and dc2, where dc1 defaults to 3 replicas and dc2 is set to 5 replicas.

cURL command (/v2)
Result

curl -L -X POST 'localhost:8082/v2/schemas/namespaces' \
-H "X-Cassandra-Token: $AUTH_TOKEN" \
-H 'Content-Type: application/json' \
-d '{
    "name": "myworld_dcs",
    "datacenters": [ {"name": "dc1"}, {"name": "dc2", "replicas": 5} ]
}'

{"name":"myworld_dcs"}

Checking namespace existence

To check if a namespaces exist, execute a Document API query with cURL to find all the namespaces:

cURL command (/v2)
Result

curl -L -X GET 'localhost:8082/v2/schemas/namespaces' \
-H "X-Cassandra-Token: $AUTH_TOKEN" \
-H 'Content-Type: application/json'

{"data":[{"name":"system_distributed"},{"name":"system"},
{"name":"data_endpoint_auth"},{"name":"system_schema"},{"name":"myworld"},
{"name":"stargate_system"},{"name":"system_auth"},{"name":"system_traces"}]}

To get a particular namespace, specify the namespace in the URL:

cURL command (/v2)
Result

curl -X GET 'localhost:8082/v2/schemas/namespaces/myworld' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{"data":{"name":"myworld"}}

Deleting a namespace

Send a DELETE request to /v2/schemas/namespaces/{namespace_name} to delete a namespace. All collections and documents will be deleted along with the namespace.

curl -L -X DELETE 'localhost:8082/v2/schemas/namespaces/myworld' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

Interacting with data stored in collections

Writing documents

All data written with the Document API is stored as JSON documents stored in collections.

For more information about the database design of the Document API, see the blog post on the Documents API.

Add document specifying a collection name

First, let’s add a document to a specified collection. Send a POST request to /v2/namespaces/{namespace_name}/collections/{collections_name} to add data to the collection fitness. The data is passed in the JSON body.

cURL command (/v2)
Result

curl --location \
--request POST 'localhost:8082/v2/namespaces/myworld/collections/fitness' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
  "id": "some-stuff",
  "other": "This is nonsensical stuff."
}'

{"documentId":"3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b"}

Notice that the document-id returned is a UUID if not specified.

Add document specifying collection name and document-id

Next, let’s add a document to a specified collection, but specify the document-id. Send a PUT request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id} to add data to the collection Janet. The document-id can be any string. The data is passed in the JSON body.

cURL command (/v2)
Result

curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
  "firstname": "Janet",
  "lastname": "Doe",
  "email": "janet.doe@gmail.com",
  "favorite color": "grey"
}'

{"documentId":"Janet"}

Note the difference between using POST and PUT. The POST request is used to insert new documents when you want the system to auto-generate the documentId. The PUT request is used to insert a new document when you want to specify the documentId.

PUT requests can also be used to update an existing document. Let’s look at those examples, next.

Inserting a sub-document

You can also insert documents that have nested values, or sub-documents. Send a PUT request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id} to add data to the existing collection. The data is passed in the JSON body.

cURL command (/v2)
Result

curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "firstname": "Joey",
    "lastname": "Doe",
    "weights": {
      "type": "bench press",
      "weight": 150,
      "reps": 15
  }
}'

{"documentId":"Joey"}

Reading documents

Retrieving a specified document

Let’s check that the data was inserted for a particular document. Send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id} to retrieve the document:

cURL command (/v2)
Result

curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
	"documentId":"3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b",
	"data":{
		"id":"some-stuff",
		"other":"This is nonsensical stuff."
	}
}

It is possible to get a value for a particular field in a document using one of two methods, either a where clause or a document-path. These methods can retrieve information from a document or a sub-document.

Retrieving a document using a where clause

Now let’s search for a particular document using a where clause. Send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}?{where-clause} to get the same information:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"firstname":\{"$eq":"Janet"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
	"data":{
		"Janet":{
			"email":"janet.doe@gmail.com",
			"favorite color":"grey",
			"firstname":"Janet",
			"lastname":"Doe"
		}
	}
}

Note that the where clause must be url encoded, so curly brackets are escaped with \ and spaces must be replaced with %20. Also, the full document is returned, as opposed to the value of the field specified in the {document-path} like the next command.

You can also search with a multiple where clause. Send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}?{where-clause} to get the same information:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"firstname":\{"$eq":"Janet"\},"lastname":\{"$eq":"Doe"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
	"data":{
		"Janet":{
			"email":"janet.doe@gmail.com",
			"favorite color":"grey",
			"firstname":"Janet",
			"lastname":"Doe"
		}
	}
}

You can also retrieve documents using a WHERE clause that searches sub-documents:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.type":\{"$eq":"bench%20press"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
	"data":{
		"Joey":{
			"firstname":"Joey",
			"lastname":"Doe",
			"weights":{
				"reps":15,
				"type":"bench press",
				"weight":150
			}
		}
	},
	"pageState":null
}

Multiple where can be used in a variety of cases. Here, a numerical value between to values is sought:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.reps":\{"$gt":12\},"weights.reps":\{"$lt":20\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "data": {
    "Joey": {
      "firstname": "Joey",
      "lastname": "Doe",
      "weights": {
        "reps": 15,
        "type": "bench press",
        "weight": 150
      }
    }
  }
}

Retrieving all documents

Let’s check that the document was inserted. Send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name} to retrieve all the documents:

cURL command (/v2)
Result

curl --location \
--request GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?page-size=3' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "data":{
     "Joey":{
	"firstname":"Joey",
	"lastname":"Doe",
	"weights":{
		"reps":15,
		"type":"bench press",
		"weight":150
	}
     },
     "Janet":{
	"email":"janet.doe@gmail.com",
	"favorite color":"grey",
	"firstname":"Janet",
	"lastname":"Doe"
     },
     "3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b":{
	"id":"some-stuff",
	"other":"This is nonsensical stuff."
     }
  }
}

The page-size parameter is included to get all the documents, rather than the last inserted document.

Retrieving all documents with Paging

For large collections, you can page a subset of the results. In this example, page-size is set to 5. Send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}page-size=5 to retrieve the first five documents (a page) in the collection:

cURL command (/v2)
Result

curl --location \
--request GET 'localhost:8082/v2/namespaces/myworld/collections/books?page-size=5' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "pageState":"JGNlYjczMDc5LTE1NTItNGQyNS1hM2ExLWE2MzgxNWVlYTAyMADwf_____B_____",
  "data":{
    "58b12778-0efd-466e-89bf-66a4c99adee1":{"author":"Tom Clancy","title":"Red Rabbit"},
    "77fe8690-f8d4-43b8-b1c9-a328318d4eae":{"author":"Tom Clancy","title":"Every Man a Tiger"},
    "3177f86c-a633-4302-92a5-de4c15ab5840":{"author":"Tom Clancy","title":"Rainbow Six"},
    "cefc7117-fc73-47b8-a965-099cf3a59269":{"author":"Tom Clancy","title":"Clear and Present Danger"},
    "ceb73079-1552-4d25-a3a1-a63815eea020":{"author":"Tom Clancy","title":"The Cardinal of the Kremlin"}
  }
}

Pay close attention to the pageState value in the results. The pageState is a string representation a location in the result set. It is essentially an encoded shortcut that allows the final result set to be built from a specific point.

In order to get the next five documents, re-run the request with page-state parameter set to the first page’s pageState:

cURL command (/v2)
Result

curl --location \
--request GET 'localhost:8082/v2/namespaces/myworld/collections/books?page-size=5&page-state=JGNlYjczMDc5LTE1NTItNGQyNS1hM2ExLWE2MzgxNWVlYTAyMADwf_____B_____ \' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "pageState":"JDUxMzVmMzFiLWY4NDYtNGZiNy1iNmFmLTk2YmU2OTMyNzlmMQDwf_____B_____",
  "data":{
    "f42cc7d8-81ba-41d5-b5c9-c13f8d0009c6":{"author":"Tom Clancy","title":"Patriot Games"},
    "46df315f-7208-4859-af1c-96c4953f70c7":{"author":"Tom Clancy","title":"Without Remorse"},
    "b23af545-4fdc-4ac5-907a-7132d7429d26":{"author":"Tom Clancy","title":"The Hunt for Red October"},
    "c6d55ecd-1631-4d5c-8320-d111662f8837":{"author":"Tom Clancy","title":"Executive Orders"},
    "5135f31b-f846-4fb7-b6af-96be693279f1":{"author":"Tom Clancy","title":"The Sum of all Fears"}
  }
}

Repeat the process for following pages by adding the page-state parameter returned from each successive result set.

The page-size parameter has a default value of 1 and a maximum value of 20.

Retrieving a specific portion of a document with document-path

To find a particular value, send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path} to retrieve the type of weights Joey has done in his workout:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey/weights/type' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{"documentId":"Joey","data":"bench press"}

In this case, the sub-document weights is the document-path specified to retrieve that data about the reps, type, and weight.

cURL command (/v2)
Result

curl --location --request GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey/weights' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
	"documentId":"Joey",
	"data":{
		"reps":15,
		"type":"bench press",
		"weight":150
	}
}

Retrieving a specific portion of a document with json-path

Let’s add another record for the next example:

cURL command (/v2)
Result

curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Martha' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "firstname": "Martha",
    "lastname": "Smith",
    "weights": {
      "type": "bench press",
      "weight": 125,
      "reps": 12
  }
}'

{"documentId":"Martha"}

To find particular values, send a GET request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{json-path} to retrieve all the users that have, say, weight reps between 11 and 16:

cURL command (/v2)
Result

curl -L -X  GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.*":\{"$gt":11\},"weights.*":\{"$lt":16\}\}&page-size=3' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header "Content-Type: application/json"

{
  "data": {
    "Joey": {
      "firstname": "Joey",
      "lastname": "Doe",
      "weights": {
        "reps": 15,
        "type": "bench press",
        "weight": 150
      }
    },
    "Martha": {
      "firstname": "Martha",
      "lastname": "Smith",
      "weights": {
        "reps": 12,
        "type": "bench press",
        "weight": 125
      }
    }
  }
}

Update documents

Data changes, so often it is necessary to update an entire document.

Replace a document

Send a PATCH request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id} to replace data to the existing collection. All fields included will be changed.

cURL command (/v2)
Result

curl -L \
-X PATCH 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "firstname": "JanetLee",
    "lastname": "Doe"
}'

{"documentId":"Janet"}

A GET request will show that the data has been replaced in the document:

cURL command (/v2)
Result

curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
 --header "X-Cassandra-Token: $AUTH_TOKEN" \
 --header 'Content-Type: application/json'

{
	"documentId":"Janet",
	"data":{
		"email":"janet.doe@gmail.com",
		"favorite color":"grey",
		"firstname":"JanetLee",
		"lastname":"Doe"
	}
}

PATCH updates are upserts. If the document doesn’t exist, it will be created. If it does exist, it will be updated with the new document data.

Replace part of of a document or sub-document

It is also possible to update only part of a document. To partially update, send a PATCH request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}. In this example, we want to change just the firstname of the document:

cURL command (/v2)
Result

curl -L \
-X PATCH 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
      "firstname": "Joseph"
}'

{"documentId":"Joey"}

and a GET will show that only the weights has been changed.

cURL command (/v2)
Result

curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "documentId":"Joey",
  "data":{
    "firstname":"Joseph",
    "lastname":"Doe",
    "weights":{
      "reps":15,
      "type":"bench press",
      "weight":150
    }
  }
}

To partially update a sub-document, send a PATCH request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path} in the same manner as the last command, but including only sub-document information to change and the document-path of the sub-document. Include all fields that you wish to update.

cURL command (/v2)
Result

curl -L \
-X PATCH 'http://localhost:8082/v2/namespaces/myworld/collections/fitness/Joey/weights' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
    "reps": 10,
    "type": "squat",
    "weight": 350
}'

{"documentId":"Joey"}

and a GET will show that only the weights has been changed.

cURL command (/v2)
Result

curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

{
  "documentId":"Joey",
  "data":{
    "firstname":"Joseph",
    "lastname":"Doe",
    "weights":{
      "reps":10,
      "type":"squat",
      "weight":350
    }
  }
}

Delete a document

To delete a document, send a DELETE request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}.

curl -L \
-X DELETE 'http://localhost:8082/v2/namespaces/myworld/collections/fitness/3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

To delete a document, based on a document path, send a DELETE request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}.

curl -L -X  DELETE 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"id":\{"$eq":"some%2Dstuff"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'

Delete a collection

Send a DELETE request to /v2/schemas/namespaces/{namespace_name}/collections/{collection_name} to delete a collection. All data will be deleted along with the collection schema.

curl -L \
-X DELETE 'http://localhost:8082/v2/namespaces/myworld/collections/fitness' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'