Using the Stargate Document API
Stargate is a data gateway deployed between client applications and a database. The Document API plugin that exposes CRUD access to data stored as unstructured JSON documents in collections.
If the Document API is used with Apache Cassandra, the document indexing is accomplished with secondary indexes. If the Document API is used with DataStax Enterprise, SAI indexing is used. The blog The Stargate Cassandra Documents API describes the underlying structure used to store collections.
Prerequisites
To use Stargate you need:
-
Docker installed and running, if using Docker
-
cURL
to run REST queries
If you are looking to just get started, DataStax Astra Database-as-a-Service can get you started with no install steps. |
This image contains the Cassandra Query Language (CQL), REST, Document, GraphQL APIs, and GraphQL Playground, along with an Apache Cassandra™ 3.11 backend.
docker pull stargateio/stargate-3_11:v1.0.13
Start the Stargate container in developer mode. Developer mode removes the need to set up a separate Cassandra instance and is meant for development and testing only.
docker run --name stargate \
-p 8080:8080 \
-p 8081:8081 \
-p 8082:8082 \
-p 127.0.0.1:9042:9042 \
-d \
-e CLUSTER_NAME=stargate \
-e CLUSTER_VERSION=3.11 \
-e DEVELOPER_MODE=true \
stargateio/stargate-3_11:v1.0.13
Swagger UI for the Document API
Once you have started the docker container, you can access the Document API
in a browser at localhost:8082/swagger-ui
.
Adding parameter information, you can generate cURL
commands to execute and
display results that will return.
Using the Auth API to generate an auth token
In order to use the Stargate Document API, an authorization token must be generated to access the interface.
The step below uses cURL
to access the REST interface to generate the needed
token.
Generate an auth token
First generate an auth token that is required in each subsequent request
in the X-Cassandra-Token
header. Note the port for the auth service is 8081.
curl -L -X POST 'http://localhost:8081/v1/auth' \
-H 'Content-Type: application/json' \
--data-raw '{
"username": "cassandra",
"password": "cassandra"
}'
You should receive a token in the response.
{"authToken":"{auth-token}"}
Use the auth token
Store the auth token in an environment variable to make it easy to use with cURL
.
export AUTH_TOKEN={auth-token}
Using Postman
If you prefer, you can use Postman as a client interface for exploring the Document API (download here). You will need to change the auth token to the header, in the collection global variables. We’ve provided a Stargate Document API Postman Collection that you can import in Postman to play with the examples shown in this walkthrough.
Now you’re ready to use the Document API for CRUD operations.
Creating or dropping a namespace
In order to use the Document API, you must create the namespace as a container that will store collections, which in turn store documents. Documents can themselves hold multiple documents. Multiple collections are contained in a namespace, but a collection cannot be contained in multiple namespaces.
Only namespaces need to specifically created. Collections are specified when a document
is inserted. A optional setting, replicas
, defines
the number of data replicas the database will store for the namespace.
If no replica is defined, then for a namespace in a single datacenter cluster,
the default is 1, and for a multiple-datacenter cluster, the default is 3 for each
datacenter.
Creating a namespace
Simple namespace
Send a POST
request to /v2/schemas/namespaces
.
In this example we use myworld
for the name
, and no replicas
setting, to default to 1.
curl --location --request POST 'localhost:8082/v2/schemas/namespaces' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"name": "myworld"
}'
{"name":"myworld"}
The authorization token and the content type are passed with --header
. The
token must be identified as X-Cassandra-Token
so that cluster recognizes the token
and its value.
The specified name for the namespace is passed as JSON data using --data
.
For shorthand, cURL
can use -L
for --location
, -X
for --request
, -H
for --header
, and -d
for --data
.
Set replicas in simple namespace
To set the replicas, send a POST
request to /v2/schemas/namespaces
.
In this example we use myworld
for the name
,
and 2
for the number of data replicas
.
Namespace for multiple datacenters
For a multiple-datacenter cluster, a namespace is defined datacenters
.
Send a POST
request to /v2/schemas/namespaces
.
In this example we use myworld-dcs
for the name
, the datacenters are dc1
and dc2
,
where dc1
defaults to 3 replicas and dc2
is set to 5 replicas.
Checking namespace existence
To check if a namespaces exist, execute a
Document API query with cURL
to find all the namespaces:
curl -L -X GET 'localhost:8082/v2/schemas/namespaces' \
-H "X-Cassandra-Token: $AUTH_TOKEN" \
-H 'Content-Type: application/json'
{"data":[{"name":"system_distributed"},{"name":"system"},
{"name":"data_endpoint_auth"},{"name":"system_schema"},{"name":"myworld"},
{"name":"stargate_system"},{"name":"system_auth"},{"name":"system_traces"}]}
To get a particular namespace, specify the namespace in the URL:
Deleting a namespace
Send a DELETE
request to /v2/schemas/namespaces/{namespace_name}
to delete
a namespace. All collections and documents will be deleted along with the
namespace.
curl -L -X DELETE 'localhost:8082/v2/schemas/namespaces/myworld' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
Interacting with data stored in collections
Writing documents
All data written with the Document API is stored as JSON documents stored in collections.
For more information about the database design of the Document API, see the blog post on the Documents API. |
Add document specifying a collection name
First, let’s add a document to a specified collection.
Send a POST
request to /v2/namespaces/{namespace_name}/collections/{collections_name}
to add data to the collection fitness
.
The data is passed in the JSON body.
curl --location \
--request POST 'localhost:8082/v2/namespaces/myworld/collections/fitness' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"id": "some-stuff",
"other": "This is nonsensical stuff."
}'
{"documentId":"3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b"}
Notice that the document-id
returned is a UUID if not specified.
Add document specifying collection name and document-id
Next, let’s add a document to a specified collection, but specify the document-id
.
Send a PUT
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}
to add data to the collection Janet
.
The document-id
can be any string.
The data is passed in the JSON body.
curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"firstname": "Janet",
"lastname": "Doe",
"email": "janet.doe@gmail.com",
"favorite color": "grey"
}'
{"documentId":"Janet"}
Note the difference between using POST
and PUT
. The POST
request
is used to insert new documents when you want the system to auto-generate the
documentId. The PUT
request is used to insert a new document when you want to
specify the documentId.
PUT
requests can also be used to update an existing
document. Let’s look at those examples, next.
Inserting a sub-document
You can also insert documents that have nested values, or sub-documents.
Send a PUT
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}
to add data to the existing collection.
The data is passed in the JSON body.
curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"firstname": "Joey",
"lastname": "Doe",
"weights": {
"type": "bench press",
"weight": 150,
"reps": 15
}
}'
{"documentId":"Joey"}
Reading documents
Retrieving all documents
Let’s check that the document was inserted. Send a GET
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}
to retrieve
all the documents:
curl --location \
--request GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?page-size=3' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"data":{
"Joey":{
"firstname":"Joey",
"lastname":"Doe",
"weights":{
"reps":15,
"type":"bench press",
"weight":150
}
},
"Janet":{
"email":"janet.doe@gmail.com",
"favorite color":"grey",
"firstname":"Janet",
"lastname":"Doe"
},
"3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b":{
"id":"some-stuff",
"other":"This is nonsensical stuff."
}
}
}
The page-size
parameter is included to get all the documents, rather than the
last inserted document.
The pageState
is useful for pagination of the results in queries.
Retrieving a specified document
Let’s check that the data was inserted for a particular document.
Send a GET
request to /v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}
to retrieve the document:
curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"documentId":"3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b",
"data":{
"id":"some-stuff",
"other":"This is nonsensical stuff."
}
}
It is possible to get a value for a particular field in a document using one of
two methods, either a where
clause or a document-path
. These methods can
retrieve information from a document or a sub-document.
Retrieving a document using a where clause
Now let’s search for a particular document using a where
clause.
Send a GET
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}?{where-clause}
to get the same information:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"firstname":\{"$eq":"Janet"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"data":{
"Janet":{
"email":"janet.doe@gmail.com",
"favorite color":"grey",
"firstname":"Janet",
"lastname":"Doe"
}
}
}
Note that the where
clause must be url encoded, so curly brackets are escaped
with \
and spaces must be replaced with %20
.
Also, the full document is returned, as opposed to the value of the field specified in the
{document-path}
like the next command.
You can also search with a multiple where
clause.
Send a GET
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}?{where-clause}
to get the same information:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"firstname":\{"$eq":"Janet"\},"lastname":\{"$eq":"Doe"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"data":{
"Janet":{
"email":"janet.doe@gmail.com",
"favorite color":"grey",
"firstname":"Janet",
"lastname":"Doe"
}
}
}
Note that the where
clause must be url encoded, so curly brackets are escaped
with \
and spaces must be replaced with %20
.
Also, the full document is returned, as opposed to the value of the field specified in the
{document-path}
like the next command.
You can also retrieve documents using a WHERE
clause that searches sub-documents:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.type":\{"$eq":"bench%20press"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"data":{
"Joey":{
"firstname":"Joey",
"lastname":"Doe",
"weights":{
"reps":15,
"type":"bench press",
"weight":150
}
}
},
"pageState":null
}
Multiple where
can be used in a variety of cases.
Here, a numerical value between to values is sought:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.reps":\{"$gt":12\},"weights.reps":\{"$lt":20\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"data": {
"Joey": {
"firstname": "Joey",
"lastname": "Doe",
"weights": {
"reps": 15,
"type": "bench press",
"weight": 150
}
}
}
}
Retrieving a specific portion of a document with document-path
To find a particular value, send a GET
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}
to retrieve the type of weights Joey has done in his workout:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey/weights/type' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{"documentId":"Joey","data":"bench press"}
In this case, the sub-document weights
is the
document-path
specified to retrieve that data about the reps
, type
, and
weight
.
Retrieving a specific portion of a document with json-path
Let’s add another record for the next example:
curl -L -X PUT 'localhost:8082/v2/namespaces/myworld/collections/fitness/Martha' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"firstname": "Martha",
"lastname": "Smith",
"weights": {
"type": "bench press",
"weight": 125,
"reps": 12
}
}'
{"documentId":"Martha"}
To find particular values, send a GET
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{json-path}
to retrieve all the users that have, say, weight reps between 11 and 16:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"weights.*":\{"$gt":11\},"weights.*":\{"$lt":16\}\}&page-size=3' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header "Content-Type: application/json"
{
"data": {
"Joey": {
"firstname": "Joey",
"lastname": "Doe",
"weights": {
"reps": 15,
"type": "bench press",
"weight": 150
}
},
"Martha": {
"firstname": "Martha",
"lastname": "Smith",
"weights": {
"reps": 12,
"type": "bench press",
"weight": 125
}
}
}
}
Update documents
Data changes, so often it is necessary to update an entire document.
Replace a document
Send a PATCH
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}
to replace data to the existing collection. All fields included will be changed.
curl -L \
-X PATCH 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"firstname": "JanetLee",
"lastname": "Doe"
}'
{"documentId":"Janet"}
A GET
request will show that the data has been replaced in the document:
curl -L -X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Janet' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"documentId":"Janet",
"data":{
"email":"janet.doe@gmail.com",
"favorite color":"grey",
"firstname":"JanetLee",
"lastname":"Doe"
}
}
PATCH updates are upserts. If the document doesn’t exist, it will be created.
If it does exist, it will be updated with the new document data.
|
Replace part of of a document or sub-document
It is also possible to update only part of a document. To partially update, send
a PATCH
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}
.
In this example, we want to change just the firstname of the document:
curl -L \
-X PATCH 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"firstname": "Joseph"
}'
{"documentId":"Joey"}
and a GET
will show that only the weights
has been changed.
curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"documentId":"Joey",
"data":{
"firstname":"Joseph",
"lastname":"Doe",
"weights":{
"reps":15,
"type":"bench press",
"weight":150
}
}
}
To partially update a sub-document, send a PATCH
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}
in the same manner as the last command, but including only sub-document
information to change and the document-path
of the sub-document. Include
all fields that you wish to update.
curl -L \
-X PATCH 'http://localhost:8082/v2/namespaces/myworld/collections/fitness/Joey/weights' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json' \
--data '{
"reps": 10,
"type": "squat",
"weight": 350
}'
{"documentId":"Joey"}
and a GET
will show that only the weights
has been changed.
curl -L \
-X GET 'localhost:8082/v2/namespaces/myworld/collections/fitness/Joey' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
{
"documentId":"Joey",
"data":{
"firstname":"Joseph",
"lastname":"Doe",
"weights":{
"reps":10,
"type":"squat",
"weight":350
}
}
}
Delete a document
To delete a document, send a DELETE
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}
.
curl -L \
-X DELETE 'http://localhost:8082/v2/namespaces/myworld/collections/fitness/3ffc7ae6-c42d-46de-b52b-b5e77ae6a87b' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
To delete a document, based on a document path, send a DELETE
request to
/v2/namespaces/{namespace_name}/collections/{collections_name}/{document-id}/{document-path}
.
curl -L -X DELETE 'localhost:8082/v2/namespaces/myworld/collections/fitness?where=\{"id":\{"$eq":"some%2Dstuff"\}\}' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'
Delete a collection
Send a DELETE
request to /v2/schemas/namespaces/{namespace_name}/collections/{collection_name}
to delete a collection. All data will be deleted along with the collection schema.
curl -L \
-X DELETE 'http://localhost:8082/v2/namespaces/myworld/collections/fitness' \
--header "X-Cassandra-Token: $AUTH_TOKEN" \
--header 'Content-Type: application/json'