If you've recently added Copado Essentials Data Deploy to your DevOps toolkit, you might be wondering what your next steps are. Well, in this article we'll help to outline what it takes to get started deploying data up to Production, or down to your Dev environments. What we'll cover below will take us through downloading and importing templates, configuring those templates for your specific use, what you'd need to do differently to seed a sandbox, and how to deploy your data. For tips and tricks on how to ensure a successful deployment, check out our Data Deploy Best Practices article.

Downloading and Importing Templates:

If you've purchased Data Deploy for the use of moving reference data, the first step you'll need to take is to visit the Copado Success website to download the Data Templates for your managed package (long gone are the days of using Data Loader and spreadsheets to maintain your object relationships!). After creating a Copado Community account, navigate to the DevOps Exchange to select the suite of templates that apply to you (whether that be for CPQ, nCino, Conga, or Veeva), then simply download these JSON files for import into your Copado Essentials account. These JSON files will show up as individual files in your local Downloads folder, but note that each JSON file will generate the main object template and all related object sub-templates upon import.

To import these templates, login to your Copado Essentials account, select "Data Templates" from the dropdown under the Data tab, and click the "Import Data Templates" button on the top right. From here, you can choose to import any selection of your templates, after which you'll see a template for the main object and sub-templates for all child and parent objects referenced by the main object template. These templates are imported in an "Active" state for ease of use, but can be deactivated at any time.

Once your templates have been imported you'll be able to customize them to fit your specific use case. By default, Object Fields, Parent Objects, Child Objects, and Parameters are all pre-selected to accommodate out of the box use, but you're free to add or subtract any fields or objects as they relate to your needs. As a precaution, within the Object Fields tab we highly recommend utilizing the External ID field (or the Lookup Key for nCino users) to ensure that an upsert takes place, so duplicates aren't created in your target environment during the data deployment process. This can be utilized by adding an External ID field to all objects referenced within your template and sub-templates, followed by selecting the field as shown below.

If you don't have any values within your External ID field, we can easily create a value for you by using the Auto Generate feature located in the "Field Content Update" dropdown on the right, then allowing you to select up to 3 object fields to hash for a unique value.

When it comes to selecting Parent and Child objects to be included in the templates, our pre-built templates come with all of the standard relationships for CPQ, Conga, nCino, and Veeva already preconfigured. However, you have the option to adjust these for any purpose that you need. To do so, just navigate to the Parent Object or Child Object tab to choose the objects you'd like to select or deselect, along with assigning a sub-template for any object added.

As a way to further customize these templates, you do have the option to apply a filter to the main object using the Main Object Filter tab. Here, you can add parameters and filter logic to specify which records you want deployed. When defining parameters, we recommend restricting this to only the top level object in your template, as adding parameters in sub-templates will significantly decrease the number records that fit your specified criteria. Parameters can be set for any field within the main object, and for more unique filtering situations, you can use a custom filter to reference cross-object. Templates can be filtered at the deployment level as well, which we discuss here.

Since templates can be shared for modification and use, filters set on the owners template may not apply to each team members use case. For these situations, it's advantageous to use the "Create My Own Filters" option to take advantage of the relationships defined in the template, but define your unique set of filter criteria. These personalized filters will override the owner's filters upon deployment.

Once your main object template has been configured to your use case, you can review the relationships visually by using the Relationship Diagram tab. This viewer will present the same information that you specified on previous tabs in a much more visually appealing way, helping to portray the relationships as they are currently defined, as well as indicating which objects have filters and files (see Sandbox Seeding Basics) attached to them. Once the template has been tailored to your needs, click "Save" for immediate use.

Templates that are active can be shared amongst your team for ease of use and customization, based on the permissions granted. Sharing with "manage" permission will allow team members to view, edit, or delete a template, where as keeping the access on "view" will only allow a team member to view the template details. Both levels of access will allows team members to use the template for a Data Deployment by using the "Include Team Template" option on a Data Deployment, discussed here.

If you've purchased Data Deploy for the purpose of seeding test data into Developer environments, then you'll likely need to create your own Data Templates. To get started creating a template, login to your Copado Essentials account, select "Data Templates" from the dropdown under the Data tab, and click the "New Data Template" button on the top right.

The first tab you'll be directed to is the Template Details tab. Here, you can define the source org of your template (which is only used to analyze the schema of your objects--this is not necessarily the org your data will come from) as well as the Main Object. You're free to select any standard or custom object as the main object for your template. In addition to specifying the source org and main object, you can define if you want your deployment to proceed in the event of an error, and how you want any attachments to be handled during a deployment (inserted, replaced, removed, or none of the above).

As detailed above, the Object Fields, Parent Objects, and Child Objects tabs can be used to create a web of relationships for records you want to be deployed from your source org to your target org, however, we strongly encourage taking a look at our Data Deploy Best Practices when constructing your templates for use.

Within the Object Fields tab, we highly recommend utilizing the External ID field to ensure that an upsert takes place, so duplicates aren't created in your target environment during the data deployment process. This can be utilized by adding an External ID field to all objects referenced within your template and sub-templates. To select Parent Objects and Child Objects, navigate to the corresponding tab to view the auto-populated list based on the Main Object selected. For every object that is selected on either of these tabs, you will need to create a new sub-template (or select one that was created previously) to specify which fields of a Parent or Child object you'd like to deploy.

Once your relationships have been defined, navigate to the Main Object Filters tab to specify parameters, filter logic, and record limits. As described above, personalized filters can be applied to a template that has been shared with you, and this will override the preset filters for a deployment, but not overwrite them. You can deploy up to 40,000 records using a data template with one main and no related objects, and up to 8,000 records using a data template with one main and many related objects. As a final step, you can review your template using the Relationship Diagram to ensure the relationships shown match what you've intended. Once you're satisfied with your template, click "Save" and "Activate" on your template for immediate use.

Once saved, templates that are "active" can be selected to use in a Data Deployment, where any relationships that you've defined will be maintained and deployed between your source and target environments. As we've discussed above, "active" templates can be shared amongst your team for ease of use and customization, based on the permissions granted. Sharing with "manage" permission will allow team members to view, edit, or delete a template, where as keeping the access on "view" will only allow a team member to view the template details. Both levels of access will allows team members to use the template for a Data Deployment by using the "Include Team Template" option on a Data Deployment, discussed in the section below.

After defining your templates, the last step is to deploy your data. To ensure a successful deployment, there are two mandatory pre-deployment actions. First, the metadata within your source and target environments need to match, so a metadata comparison between environments is recommended. This comparison can be accomplished by setting up a mock metadata Deployment and choosing the same source and target orgs that will be used in the Data Deployment, then comparing the fields and objects that data will be deployed between. Second, it is best practice to disable Salesforce Triggers, Workflows and Validations when working with Copado Data Deployments. These processes may adversely affect the insertion or update of a record, so temporarily disabling them is recommended. For more information on how to disable these processes, see this Copado Enterprise help article. Additional information on Data Deploy Best Practices can be found here.

When you're ready to deploy your data, click "Data Deployments" from the dropdown under the Data tab in the top banner, followed by "New Data Deployment".

On this this screen, you can define your source org, target org, and data template to be used, as well as sync a Jira/Azure Boards ticket along with listing any note to be associated with the deployment.

By default, templates visible here are those that are set to "Active" within your own instance of Essentials, however there may be occasions where a teammates template needs to be used. In these situations, you can easily access team members' templates by checking the "Include team templates" field, as shown below. In doing so, you'll be able to see any active template across your entire team for use.

After the deployment has been defined, click "Save", and you'll be brought to a summary screen showing the details of the current data deployment. On this screen, you'll be able to see the ALM ticket and note associated with this deployment, source and target for the deployment, the template selected for use, and the option to customize your deployment even further using additional filtering at the deployment level. Filtering applied at the deployment level is not saved to the template and only exists for the deployment, making this a very flexible option for adding additional parameters for specific use cases.

Once your filters have been set, click the Deploy button to initiate your data deployment. After your deployment has completed, you'll be able to view details relating to the deployment by clicking the blue button beneath the deployment status.

On the view details pop-up, you can see how many records were deployed, and from which objects, to double check your object relationships were maintained as intended. If there were any failed record deployments, you could see those indicated in the center column, and get additional information by downloading a detailed CSV of the deployment using the "click here" button.

Currently, the importing and exporting of Data Sets is not supported with Copado Essentials Data Deploy, however this is on the roadmap for future releases. This feature is currently available with Copado Enterprise, for which you can review the documentation here.

Data Deploy is a powerful add-on that can help to simplify and automate your data migrations, all while complementing the quick metadata deployments of Copado Essentials. If you need any additional resources while setting up and using Data Deploy, feel free to reach out to us in-app using the chat icon on the bottom right for access to 24/7 support.