Insert documents (Python)
Inserts multiple documents into a collection.
Documents are stored in collections. They represent a single row or record of data in Astra DB Serverless databases. For more information, see About collections with the Data API (Python).
If the collection is vector-enabled, pregenerated vector embeddings can be included by using the reserved $vector field for each document.
If the collection has vectorize enabled, vector embeddings can be automatically generated from text specified in the reserved $vectorize field for each document.
You can later use the $vector or $vectorize field to perform a vector search or hybrid search.
If the collection has lexical enabled, use the reserved $lexical field to store a string to index for lexicographical matching and the lexical search component of hybrid search.
Alternatively, you can use the $hybrid shorthand to populate the $vectorize and $lexical fields.
|
Ready to write code? See the examples for this method to get started. If you are new to the Data API, check out the quickstart. |
Result
Inserts the specified documents and returns a CollectionInsertManyResult object that includes the IDs of the inserted documents and details about the operation.
The ID value depends on the ID type. For more information, see Document IDs (Python).
Example response:
CollectionInsertManyResult(inserted_ids=[
"3f557bef-fd53-47ea-957b-effd53c7eaec",
101,
"132ffr343"
], raw_results=...)
Parameters
Use the insert_many method, which belongs to the astrapy.Collection class.
Method signature
insert_many(
documents: Iterable[Dict[str, Any]],
*,
ordered: bool,
chunk_size: int,
concurrency: int
general_method_timeout_ms: int,
request_timeout_ms: int,
timeout_ms: int,
) -> CollectionInsertManyResult
| Name | Type | Summary |
|---|---|---|
|
|
An iterable of dictionaries, with each dictionary describing a document to insert. A document can contain user-defined and reserved fields. User-defined field names can be any non-empty sequence of Unicode characters, with the following exceptions:
Reserved fields are tied to specific functionality. Include the following reserved fields in your documents, if applicable:
For examples, see Examples. |
|
|
Optional.
Whether the insertions must be processed sequentially.
If For an example, see Insert documents and specify insertion behavior. Default: |
|
|
Optional. The number of documents to include in a single API request. DataStax recommends leaving this parameter unspecified to use the system default. For an example, see Insert documents and specify insertion behavior. Maximum: Default: |
|
|
Optional. The maximum number of concurrent requests to the API at a given time. If For an example, see Insert documents and specify insertion behavior. Default: |
|
|
Optional. The maximum time, in milliseconds, that the whole operation, which might involve multiple HTTP requests, can take. Default: The default value for the collection. This default is 30 seconds unless you specified a different default when you initialized the This parameter is aliased as |
|
|
Optional. The maximum time, in milliseconds, that the client should wait for each underlying HTTP request. Default: The default value for the collection. This default is 10 seconds unless you specified a different default when you initialized the |
Examples
The following examples demonstrate how to insert multiple documents into a collection.
Insert documents
The documents can have different structures.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents into the collection
result = collection.insert_many(
[
{
"name": "Jane Doe",
"age": 42,
},
{
"nickname": "Bobby",
"color": "blue",
"foods": ["carrots", "chocolate"],
},
]
)
Insert documents with vector embeddings
Use the reserved $vector field to insert documents with pregenerated vector embeddings.
All embeddings in the collection should use the same provider, model, and dimensions. Mismatched embeddings can cause inaccurate vector searches.
The $vector field is only supported for vector-enabled collections.
For more information, see Create a collection that can store vector embeddings and $vector in collections (Python).
You may also insert a mix of documents with and without the $vector field.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents to the collection
# The following also demonstrates use of both plain lists and DataAPIVector
result = collection.insert_many(
[
{"name": "Jane Doe", "age": 42, "$vector": [0.08, -0.62, 0.39]},
{
"nickname": "Bobby",
"$vector": [0.12, 0.53, 0.32],
},
]
)
Insert documents and generate vector embeddings
Use the reserved $vectorize field to generate a vector embedding automatically. The value of $vectorize can be any string.
The $vectorize field is only supported for collections that have vectorize enabled.
For more information, see Create a collection that can automatically generate vector embeddings and $vectorize in collections (Python).
You may also insert a mix of documents with and without the $vectorize field.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents into the collection
result = collection.insert_many(
[
{
"name": "Jane Doe",
"age": 42,
"$vectorize": "Text to vectorize for this document",
},
{
"nickname": "Bobby",
"$vectorize": "Text to vectorize for this document",
},
]
)
Insert documents for retrieval with hybrid search
|
Hybrid search and reranking are currently in public preview. Development is ongoing, and the features and functionality are subject to change. Astra DB Serverless, and the use of such, is subject to the DataStax Preview Terms. |
If you plan to use hybrid search to find documents, each document must have both the $lexical field and the $vector field populated.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents
result = collection.insert_many(
[
{
"name": "Jane Doe",
"$vector": [0.08, -0.62, 0.39],
"$lexical": "An author who writes SciFi and fantasy novels.",
},
{
"name": "Mary Day",
"$vectorize": "An athlete who loves biking, hiking, running, and swimming in the outdoors",
"$lexical": "She shares her love of triathlons by coaching kids after school.",
},
{
"name": "Bobby",
"$hybrid": "A software developer who enjoys managing databases",
},
]
)
Insert documents for retrieval with lexicographical matching
|
Lexicographical matching is currently in public preview. Development is ongoing, and the features and functionality are subject to change. Astra DB Serverless, and the use of such, is subject to the DataStax Preview Terms. |
If you plan to use lexicographical matching to find documents, each document must have the $lexical field populated.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents
result = collection.insert_many(
[
{
"name": "Jane Doe",
"$lexical": "An author who writes SciFi and fantasy novels.",
},
{
"name": "Mary Day",
"$lexical": "An active hiker, runner, and triathlete who loves the outdoors.",
},
]
)
Insert documents and specify the IDs
The Python client provides the UUID and ObjectId classes to use and generate identifiers.
from astrapy import DataAPIClient
from astrapy.ids import UUID, ObjectId
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents into the collection
result = collection.insert_many(
[
{
"name": "Melissa",
"_id": ObjectId("6672e1cbd7fabb4e5493916f"),
},
{
"name": "Jess",
"_id": UUID("1ef2e42c-1fdb-6ad6-aae4-e84679831739"),
},
{
"name": "Jane",
"_id": 1,
},
{
"name": "Bobby",
"_id": "b_023",
},
]
)
Insert documents and specify insertion behavior
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents into the collection
result = collection.insert_many(
[
{
"name": "Jane Doe",
"age": 42,
},
{
"nickname": "Bobby",
"color": "blue",
"foods": ["carrots", "chocolate"],
},
],
chunk_size=2,
concurrency=2,
ordered=False,
general_method_timeout_ms=1000,
)
Insert documents with a binary field
You can insert binary data as a Base64-encoded string with $binary or as a bytes value.
The Python client returns binary data received from the Data API as a bytes value, even if it was inserted as a Base64-encoded string.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert a document with binary fields
result = collection.insert_many(
[
{
"exampleBinary": {"$binary": "PfvnbT7peNU/Sfvn"},
"anotherExampleBinary": b"=\xfb\xe7m>\xe9x\xd5?I\xfb\xe7",
}
]
)
Insert documents with nested fields
Although you can use dot notation in a filter to find a document, you cannot use dot notation to insert a document. To specify nested fields in the inserted document, you must build a map, list, or set.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Insert documents into the collection
result = collection.insert_many(
[
{
"title": "Hidden Shadows of the Past",
"genres": [
"Biography",
"Graphic Novel",
"Dystopian",
"Drama",
],
"metadata": {
"isbn": "978-1-905585-40-3",
"language": "French",
"edition": "Anniversary Edition",
},
},
{
"title": "Bake a Dozen",
"genres": ["Biography", "Fiction"],
"metadata": {
"isbn": "342-2-875587-50-2",
"language": "English",
"edition": "Illustrated Edition",
},
},
]
)
Client reference
For more information, see the client reference.