Integration Consideration - To Merge or Not To Merge

If there is MERGE requirement, then Delta Lake is the only option. Otherwise, the Databricks target table does not have to be Delta.

Integration Scenario - Delta Lake

Leveraging Delta Lake, users could obtain a replica of the source SAP objects in Databricks data lake for data scientists to explore. The incremental delta records from SAP are merged into Delta Lake automatically by AecorSoft Data Integrator.

Integration Scenario - Non-Delta

If there is no Merge requirement, then either Delta table or non-Delta table could be considered. For example, the requirement for full refresh does not require Delta. If there is requirement to keep the SAP delta records change log history by appending records to the table, then the non-Delta table would also work.

Loading data into non-Delta table is relatively faster than loading into Delta table.