If you've recently added Copado Essentials Data Deploy to your DevOps toolkit, you might be wondering what your next steps are. Well, in this article we'll help to outline what it takes to get started deploying data up to Production, or down to your Dev environments. What we'll cover below will take us through downloading and importing templates, configuring those templates for your specific use, what you'd need to do differently to seed a sandbox, and how to deploy your data. For tips and tricks on how to ensure a successful deployment, check out our Data Deploy Best Practices article.
Template Layout:
Before diving into the setup process, it's helpful to understand the structure of a Data Deploy template. Each template consists of six main sections that work together to define what data gets deployed and how it's handled during the migration process.
The Template Details tab serves as the foundation, where you'll define your source org (used for schema analysis), select your main object, and configure how errors and attachments are handled during deployment. When selecting your Template Source Org, it's recommended to use the org that contains the most complete and up-to-date schema for your objects, as this ensures all available fields and relationships are visible when configuring your template. The Match Owners option allows you to maintain record ownership during deployment - when enabled, Copado will attempt to match user records between source and target orgs based on username or email. However, be aware that if an owner cannot be found in the source environment (such as inactive users), the entire deployment may fail.
The Object Fields tab is where you'll specify which fields from your main object should be included in the deployment. Here you'll see several important columns: the SF Att column displays field attributes where XID indicates the field has External ID capability, and a red asterisk (*) denotes required fields that must be populated for successful record creation. The Use as External ID column allows you to designate which field should serve as your External ID - a critical component that ensures upserts take place rather than creating duplicate records in your target environment. Additionally, you can leverage Field Content Update options here to scramble sensitive data or replace values as needed for compliance purposes.
When it comes to maintaining relationships, the Parent Objects and Child Objects tabs allow you to include related data through lookup and master-detail relationships. In these sections, the red asterisk (*) in the SF Att column indicates required relationship fields that must be populated for the related object records to be created successfully. Each related object you select will require its own sub-template to specify which fields should be deployed, ensuring that your complex object relationships are preserved during the migration.
The Main Object Filter tab gives you control over which records actually get deployed by allowing you to set parameters, filter logic, and record limits. The record deployment limits are determined by template complexity: you can deploy up to 40,000 records with a simple template containing only the main object and no related objects, or up to 8,000 records when including related objects due to the additional processing overhead required to maintain relationships. For more complex scenarios, you can create personalized filters that override the template owner's settings without permanently modifying the template itself.
Finally, the Relationship Diagram tab provides a visual representation of your template structure, making it easy to verify that your object relationships are configured as intended and to identify which objects have filters or file attachments applied. You can click directly on any object in the diagram to open and edit the corresponding template, providing a convenient way to navigate between your main template and sub-templates while reviewing your configuration.
Downloading and Importing Templates:
Now that we've reviewed template structure, if you've purchased Data Deploy for the use of moving reference data, the first step you'll need to take is to visit the Copado Success website to download the Data Templates for your managed package (long gone are the days of using Data Loader and spreadsheets to maintain your object relationships!). After creating a Copado Community account, navigate to the DevOps Exchange to select the suite of templates that apply to you (whether that be for CPQ, nCino, Conga, or Veeva), then simply download these JSON files for import into your Copado Essentials account. These JSON files will show up as individual files in your local Downloads folder, but note that each JSON file will generate the main object template and all related object sub-templates upon import.
To import these templates, login to your Copado Essentials account, select "Data Templates" from the dropdown under the Data tab, and click the "Import Data Templates" button on the top right. From here, you can choose to import any selection of your templates, after which you'll see a template for the main object and sub-templates for all child and parent objects referenced by the main object template. These templates are imported in an "Active" state for ease of use, but can be deactivated at any time.
Configuring Pre-Built Data Templates:
Once your templates have been imported you'll be able to customize them to fit your specific use case. By default, Object Fields, Parent Objects, Child Objects, and Parameters are all pre-selected to accommodate out of the box use, but you're free to add or subtract any fields or objects as they relate to your needs. As a precaution, within the Object Fields tab we highly recommend utilizing the External ID field (or the Lookup Key for nCino users) to ensure that an upsert takes place, so duplicates aren't created in your target environment during the data deployment process. This can be utilized by adding an External ID field to all objects referenced within your template and sub-templates, followed by selecting the field as shown below.
Note: If an External ID is not used, Data Deploy will insert records regardless of if an existing record is present in the target environment.
If you don't have any values within your External ID field, we can easily create a value for you by using the Auto Generate feature located in the "Field Content Update" dropdown on the right, then allowing you to select up to 3 object fields to hash for a unique value.
When it comes to selecting Parent and Child objects to be included in the templates, our pre-built templates come with all of the standard relationships for CPQ, Conga, nCino, and Veeva already preconfigured. However, you have the option to adjust these for any purpose that you need. To do so, just navigate to the Parent Object or Child Object tab to choose the objects you'd like to select or deselect, along with assigning a sub-template for any object added.
As a way to further customize these templates, you do have the option to apply a filter to the main object using the Main Object Filter tab. Here, you can add parameters and filter logic to specify which records you want deployed. When defining parameters, we recommend restricting this to only the top level object in your template, as adding parameters in sub-templates will significantly decrease the number records that fit your specified criteria. Parameters can be set for any field within the main object, and for more unique filtering situations, you can use a custom filter to reference cross-object. Templates can be filtered at the deployment level as well, which we discuss here.
Since templates can be shared for modification and use, filters set on the owners template may not apply to each team members use case. For these situations, it's advantageous to use the "Create My Own Filters" option to take advantage of the relationships defined in the template, but define your unique set of filter criteria. These personalized filters will override the owner's filters upon deployment.
Once your main object template has been configured to your use case, you can review the relationships visually by using the Relationship Diagram tab. This viewer will present the same information that you specified on previous tabs in a much more visually appealing way, helping to portray the relationships as they are currently defined, as well as indicating which objects have filters and files (see Sandbox Seeding Basics) attached to them. Once the template has been tailored to your needs, click "Save" for immediate use.
Templates that are active can be shared amongst your team for ease of use and customization, based on the permissions granted. Sharing with "manage" permission will allow team members to view, edit, or delete a template, where as keeping the access on "view" will only allow a team member to view the template details. Both levels of access will allows team members to use the template for a Data Deployment by using the "Include Team Template" option on a Data Deployment, discussed here.
Sandbox Seeding Basics:
If you've purchased Data Deploy for the purpose of seeding test data into Developer environments, then you'll likely need to create your own Data Templates. To get started creating a template, login to your Copado Essentials account, select "Data Templates" from the dropdown under the Data tab, and click the "New Data Template" button on the top right.
The first tab you'll be directed to is the Template Details tab. Here, you can define the source org of your template (which is only used to analyze the schema of your objects--this is not necessarily the org your data will come from) as well as the Main Object. You're free to select any standard or custom object as the main object for your template. In addition to specifying the source org and main object, you can define if you want your deployment to proceed in the event of an error, and how you want any attachments to be handled during a deployment (inserted, replaced, removed, or none of the above).
As detailed above, the Object Fields, Parent Objects, and Child Objects tabs can be used to create a web of relationships for records you want to be deployed from your source org to your target org, however, we strongly encourage taking a look at our Data Deploy Best Practices when constructing your templates for use.
Within the Object Fields tab, we highly recommend utilizing the External ID field to ensure that an upsert takes place, so duplicates aren't created in your target environment during the data deployment process. This can be utilized by adding an External ID field to all objects referenced within your template and sub-templates. To select Parent Objects and Child Objects, navigate to the corresponding tab to view the auto-populated list based on the Main Object selected. For every object that is selected on either of these tabs, you will need to create a new sub-template (or select one that was created previously) to specify which fields of a Parent or Child object you'd like to deploy.
Once your relationships have been defined, navigate to the Main Object Filters tab to specify parameters, filter logic, and record limits. As described above, personalized filters can be applied to a template that has been shared with you, and this will override the preset filters for a deployment, but not overwrite them. You can deploy up to 40,000 records using a data template with one main and no related objects, and up to 8,000 records using a data template with one main and many related objects. As a final step, you can review your template using the Relationship Diagram to ensure the relationships shown match what you've intended. Once you're satisfied with your template, click "Save" and "Activate" on your template for immediate use.
Once saved, templates that are "active" can be selected to use in a Data Deployment, where any relationships that you've defined will be maintained and deployed between your source and target environments. As we've discussed above, "active" templates can be shared amongst your team for ease of use and customization, based on the permissions granted. Sharing with "manage" permission will allow team members to view, edit, or delete a template, where as keeping the access on "view" will only allow a team member to view the template details. Both levels of access will allows team members to use the template for a Data Deployment by using the "Include Team Template" option on a Data Deployment, discussed in the section below.
Deploying Data:
After defining your templates, the last step is to deploy your data. To ensure a successful deployment, there are two mandatory pre-deployment actions. First, the metadata within your source and target environments need to match, so a metadata comparison between environments is recommended. This comparison can be accomplished by setting up a mock metadata Deployment and choosing the same source and target orgs that will be used in the Data Deployment, then comparing the fields and objects that data will be deployed between. Second, it is best practice to disable Salesforce Triggers, Workflows and Validations when working with Copado Data Deployments. These processes may adversely affect the insertion or update of a record, so temporarily disabling them is recommended. For more information on how to disable these processes, see this Copado Enterprise help article. Additional information on Data Deploy Best Practices can be found here.
When you're ready to deploy your data, click "Data Deployments" from the dropdown under the Data tab in the top banner, followed by "New Data Deployment".
On this this screen, you can define your source org, target org, and data template to be used, as well as sync a Jira/Azure Boards ticket along with listing any note to be associated with the deployment.
By default, templates visible here are those that are set to "Active" within your own instance of Essentials, however there may be occasions where a teammates template needs to be used. In these situations, you can easily access team members' templates by checking the "Include team templates" field, as shown below. In doing so, you'll be able to see any active template across your entire team for use.
After the deployment has been defined, click "Save", and you'll be brought to a summary screen showing the details of the current data deployment. On this screen, you'll be able to see the ALM ticket and note associated with this deployment, source and target for the deployment, the template selected for use, and the option to customize your deployment even further using additional filtering at the deployment level. Filtering applied at the deployment level is not saved to the template and only exists for the deployment, making this a very flexible option for adding additional parameters for specific use cases.
Once your filters have been set, click the Deploy button to initiate your data deployment. After your deployment has completed, you'll be able to view details relating to the deployment by clicking the blue button beneath the deployment status.
On the view details pop-up, you can see how many records were deployed, and from which objects, to double check your object relationships were maintained as intended. If there were any failed record deployments, you could see those indicated in the center column, and get additional information by downloading a detailed CSV of the deployment using the "click here" button.
Data Sets:
Currently, the importing and exporting of Data Sets is not supported with Copado Essentials Data Deploy, however this is on the roadmap for future releases. This feature is currently available with Copado Enterprise, for which you can review the documentation here.
Conclusion:
Data Deploy is a powerful add-on that can help to simplify and automate your data migrations, all while complementing the quick metadata deployments of Copado Essentials. If you need any additional resources while setting up and using Data Deploy, feel free to reach out to us in-app using the chat icon on the bottom right for access to 24/7 support.