Find documents (Python)
Finds documents in a collection using filter and sort clauses, including vector search.
To find documents with hybrid search, see Find and rerank documents (Python).
If you add or remove documents after starting the operation, the result might not reflect real-time changes in the data.
|
Ready to write code? See the examples for this method to get started. If you are new to the Data API, check out the quickstart. |
Result
Returns a cursor (CollectionFindCursor) for iterating over documents that match the specified filter and sort clauses.
The fields included in the returned documents depend on the subset of fields that were requested in the projection.
If requested and applicable, each document will also include a $similarity key with a numeric similarity score that represents the closeness of the sort vector and the document’s vector.
If requested when executing a vector search, the result will also include the sort vector.
You must iterate over the cursor to fetch matching documents. For details about iteration, see Iterate over found documents.
For more information about the operations available on cursors, see FindCursor.
Parameters
Use the find method, which belongs to the astrapy.Collection class.
Method signature
find(
filter: Dict[str, Any],
*,
projection: Dict[str, bool],
document_type: type,
skip: int,
limit: int,
include_similarity: bool,
include_sort_vector: bool,
sort: Dict[str, Any],
initial_page_state: str,
request_timeout_ms: int,
timeout_ms: int,
) -> CollectionFindCursor
| Name | Type | Summary |
|---|---|---|
|
|
Optional. An object that defines filter criteria using the Data API filter syntax. The method only finds documents that match the filter criteria. Filters can improve performance by reducing the number of documents that the Data API processes. You must use For a list of available filter operators and more examples, see Filter operators for collections (Python). Filters can use only indexed fields. If you apply selective indexing when you create a collection, you cannot reference non-indexed fields in a filter. For an example, see Use filters to find documents. |
|
|
Optional. Controls which fields are included or excluded in the returned document. You must use For more information, see Projections for collections (Python). Default: The default projection for the collection.
All fields prefixed with For examples, see Include only specific fields in the response and Exclude specific fields from the response. |
|
|
A formal specifier for the type checker. Use this parameter if your code is strictly typed, especially if you specify a projection. For more information, see Typing support. Default: |
|
|
Optional. The number of documents to bypass (skip) before returning documents. The API excludes the first This parameter only applies if you also explicitly specify an ascending or descending sort criterion. This parameter is not valid with vector search. For an example, see Skip documents. |
|
|
Limit the total number of documents returned.
Once For vector search, a lower limit reduces the accuracy of the search and the time required for the search. For an example, see Limit the number of documents returned. |
|
|
Optional.
Whether to include a The This parameter only applies if you use a vector search. For an example, see Include the similarity score with the result. Default: False |
|
|
Optional. Whether to include the sort vector in the response. This can be useful if you do a vector search with Because vector search is approximate, setting a lower limit increases the chance of finding a close match, but not necessarily the best match. This parameter only applies if you use a vector search. For an example, see Include the sort vector with the result. Default: False |
|
|
Optional. Sorts documents by one or more fields, or performs a vector search. You must use For more information, see Sort clauses for collections (Python). Sort clauses can use only indexed fields. If you apply selective indexing when you create a collection, you cannot reference non-indexed fields in sort queries. For vector searches, this parameter can use either |
|
|
Optional.
The Used to manually request the next page of results. This is useful for cases where an external action triggers fetching the next page of results. For an example, see Iterate over found documents. |
|
|
Optional. The maximum time, in milliseconds, that the client should wait for each underlying HTTP request. Default: The default value for the collection. This default is 10 seconds unless you specified a different default when you initialized the |
|
|
Optional.
An alias for |
Examples
The following examples demonstrate how to find documents in a collection.
Use filters to find documents
You can use a filter to find documents that match specific criteria.
For example, you can find documents with an is_checked_out value of false and a number_of_pages value less than 300.
For a list of available filter operators and more examples, see Filter operators for collections (Python).
Filters can use only indexed fields. If you apply selective indexing when you create a collection, you cannot reference non-indexed fields in a filter.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{
"$and": [
{"is_checked_out": False},
{"number_of_pages": {"$lt": 300}},
]
}
)
# Iterate over the found documents
for document in cursor:
print(document)
Use vector search to find documents
To find the documents whose $vector value is most similar to a given vector, use a sort with the vector embeddings that you want to match. For more information, see Find data with vector search.
Vector search is only available for vector-enabled collections.
For more information, see Create a collection that can store vector embeddings and $vector in collections (Python).
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find({}, sort={"$vector": [0.08, -0.62, 0.39]})
# Iterate over the found documents
for document in cursor:
print(document)
Use vector search and vectorize to find documents
To find the document whose $vector value is most similar to the $vector value of a given search string, use a sort with the search string that you want to vectorize and match. For more information, see Find data with vector search.
Vector search with vectorize is only available for collections that have vectorize enabled.
For more information, see Create a collection that can automatically generate vector embeddings and $vectorize in collections (Python).
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find({}, sort={"$vectorize": "Text to vectorize"})
# Iterate over the found documents
for document in cursor:
print(document)
Use lexicographical matching to find documents
|
Lexicographical matching is currently in public preview. Development is ongoing, and the features and functionality are subject to change. Astra DB Serverless, and the use of such, is subject to the DataStax Preview Terms. |
There are two ways to use lexicographical matching to find documents with the Data API:
-
Sort on the
$lexicalfield to find the documents whose$lexicalfield value is most relevant to a given string of space-separated keywords or terms -
Filter on the
$lexicalfield with the$matchoperator to find the documents whose$lexicalfield value is a lexicographical match to the specified string of space-separated keywords or terms
You can use these strategies together or separately.
You can only use lexicographical matching on collections that have lexical enabled. For more information, see Create a collection that supports lexicographical matching.
Documents must have the $lexical field populated to be included in lexicographical matching.
For examples, see Insert a document for retrieval with lexicographical matching and Insert documents for retrieval with lexicographical matching.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{"$lexical": {"$match": "tree hill"}},
sort={"$lexical": "tree hill grassy"},
)
# Iterate over the found documents
for document in cursor:
print(document)
Use sorting to find documents
You can use a sort clause to sort documents by one or more fields.
For more information, see Sort clauses for collections (Python).
Sort clauses can use only indexed fields. If you apply selective indexing when you create a collection, you cannot reference non-indexed fields in sort queries.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
from astrapy.constants import SortMode
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{"metadata.language": "English"},
sort={
"rating": SortMode.ASCENDING,
"title": SortMode.DESCENDING,
},
)
# Iterate over the found documents
for document in cursor:
print(document)
Use an empty filter to find all documents
To find all documents, use an empty filter.
You should avoid this if you have a large number of documents.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find({})
# Iterate over the found documents
for document in cursor:
print(document)
Include the similarity score with the result
If you use a vector search to find documents, you can also include a $similarity property for each document in the result. The $similarity value represents the closeness of the sort vector and the document’s vector.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{}, sort={"$vectorize": "Text to vectorize"}, include_similarity=True
)
# Iterate over the found documents
for document in cursor:
print(document["$similarity"])
Include the sort vector with the result
If you use a vector search to find documents, you can also include the sort vector in the result. This can be useful if you do a vector search with $vectorize, since you don’t know the sort vector in advance.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{}, sort={"$vectorize": "Text to vectorize"}, include_sort_vector=True
)
# Get the sort vector from the result
vector = cursor.get_sort_vector()
print(vector)
Include only specific fields in the response
To specify which fields to include or exclude in the returned documents, use a projection.
All fields prefixed with $ are excluded by default and will only be returned if you include them in the projection.
_id is included by default and will always be returned unless you exclude it from the projection.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{"metadata.language": "English"},
projection={"is_checked_out": True, "title": True},
)
# Iterate over the found documents
for document in cursor:
print(document)
Exclude specific fields from the response
To specify which fields to include or exclude in the returned document, use a projection.
All fields prefixed with $ are excluded by default and will only be returned if you include them in the projection.
_id is included by default and will always be returned unless you exclude it from the projection.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{"metadata.language": "English"},
projection={"is_checked_out": False, "title": False},
)
# Iterate over the found documents
for document in cursor:
print(document)
Limit the number of documents returned
Specify a limit to only fetch up to a certain number of documents.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find({"metadata.language": "English"}, limit=10)
# Iterate over the found documents
for document in cursor:
print(document)
Skip documents
You can specify a number of documents to skip (bypass) before returning documents.
You can only do this if your find explicitly includes an ascending or descending sort criterion.
You cannot do this in conjunction with vector search.
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
from astrapy.constants import SortMode
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{"metadata.language": "English"},
sort={
"rating": SortMode.ASCENDING,
"title": SortMode.DESCENDING,
},
skip=5,
)
# Iterate over the found documents
for document in cursor:
print(document)
Use filter, sort, and projection together
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
from astrapy.constants import SortMode
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{
"$and": [
{"is_checked_out": False},
{"number_of_pages": {"$lt": 300}},
]
},
sort={
"rating": SortMode.ASCENDING,
"title": SortMode.DESCENDING,
},
projection={"is_checked_out": True, "title": True},
)
# Iterate over the found documents
for document in cursor:
print(document)
Iterate over found documents
Use a for loop to iterate over the cursor. The client will periodically fetch more documents until no matching documents remain.
Alternatively, you can use the initial_page_state parameter to fetch a specific page of results.
This is useful for cases where an external action triggers fetching the next page of results.
For example, you might use this feature if you implement an infinite scroll interface or a button to load more results.
If you need a list of all results, call to_list().
However, the time and memory required for this operation depend on the number of results.
This is not recommended when you expect a large number of documents.
Example using for:
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find documents
cursor = collection.find(
{
"$and": [
{"is_checked_out": False},
{"number_of_pages": {"$lt": 300}},
]
}
)
# Iterate over the found documents
for document in cursor:
print(document)
Example using initial_page_state:
from astrapy import DataAPIClient
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Create the filter
filter = {
"$and": [
{"is_checked_out": False},
{"number_of_pages": {"$lt": 300}},
]
}
# Get the first page
cursor_1 = collection.find(filter)
page_1 = cursor_1.fetch_next_page()
results_1 = page_1.results
for document in results_1:
print(document)
pagination_state_1 = page_1.next_page_state
# Get the next page
if pagination_state_1:
cursor_2 = collection.find(
filter,
initial_page_state=pagination_state_1,
)
page_2 = cursor_2.fetch_next_page()
results_2 = page_2.results
for document in results_2:
print(document)
pagination_state_2 = page_2.next_page_state
Work with . and & in field names
You must use & to escape any . or & in field names when the field is used in a filter, sort, projection, update, or indexing clause.
Dot notation, which is used to reference nested fields, should not be escaped.
For more information, see Work with . and & in field names (Python).
For example, in the following document, you would use escaping like this: areas.r&&d, costs.price&.usd, and costs.price&.cad.
{
"areas": {
"r&d": true,
"design": false
},
"costs": {
"price.usd": 100,
"price.cad": 90
}
}
The following example uses untyped documents or rows, but you can define a client-side type for your collection to help statically catch errors. For examples, see Typing support.
from astrapy import DataAPIClient
from astrapy.constants import SortMode
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find a document
cursor = collection.find(
{
"$and": [
{"areas.r&&d": False},
{"costs.price&.usd": {"$lt": 300}},
]
},
sort={"costs.price&.usd": SortMode.ASCENDING},
projection={"areas.r&&d": True, "costs.price&.cad": True},
)
# Iterate over the found documents
for document in cursor:
print(document)
You can also use the escape_field_names function provided by the client:
from astrapy import DataAPIClient
from astrapy.constants import SortMode
from astrapy.utils.document_paths import escape_field_names
# Get an existing collection
client = DataAPIClient()
database = client.get_database(
"API_ENDPOINT", token="APPLICATION_TOKEN"
)
collection = database.get_collection("COLLECTION_NAME")
# Find a document
cursor = collection.find(
{
"$and": [
{escape_field_names("areas", "r&d"): False},
{escape_field_names("costs", "price.usd"): {"$lt": 300}},
]
},
sort={escape_field_names("costs", "price.usd"): SortMode.ASCENDING},
projection={
escape_field_names("areas", "r&d"): True,
escape_field_names("costs", "price.cad"): True,
},
)
# Iterate over the found documents
for document in cursor:
print(document)
Client reference
For more information, see the client reference.