Migrate or insert large amounts of data
When inserting large amounts of data, a programmatic approach is often more performant than the Astra Portal.
For required permissions to insert data, see Ways to insert data in Astra DB Serverless.
Use the Data API for bulk inserts
To insert a large amount of data with the Data API, use the command to insert many documents or insert many rows.
You must batch your requests to insert fewer than 100 documents or rows and fewer than 20 million characters at a time.
For Data API commands and examples, see the following:
|
If you’re new to the Data API, try the quickstart for collections or the quickstart for tables for a demo of some common operations, including inserting data from a file. |
Use CQL and drivers for bulk inserts
There are multiple strategies you can use to handle bulk inserts in CQL statements, such as batching, prepared statements, and the COPY command.
For information, see the following:
Use the DataStax Bulk Loader to insert large CSV and JSON files
You can use the DataStax Bulk Loader (DSBulk) to insert CSV and JSON files into collections and tables in Astra DB Serverless databases. For files larger than 40 MB, DataStax recommends DSBulk instead of the Astra Portal because DSBulk is more performant at this scale.
|
Your CSV or JSON file must be compatible with Astra DB and, if applicable, the table schema. For example, if you insert a CSV file into a table, the CSV file must contain the same column names and data types as the table. If you insert a JSON file exported from a database that isn’t based on Apache Cassandra®, you might need to transform the data into a format that is compatible with Astra DB before you insert the data. For more information, see [migrate-from-non-cassandra-sources]. |
DSBulk 1.11.0 or later is required to support vector data.
For installation and usage instructions, see About DataStax Bulk Loader.
Migrate data to Astra DB
DataStax offers several options to help migrate your data to Astra DB.
After migrating your data to Astra DB, your applications can connect exclusively to your Astra DB databases. Your code might not require any other changes if you already use a compatible driver and CQL statements. To get started on application development with Astra DB, see Connect to a database.
For more information, see Migrate to Astra DB Serverless.
