Use Azure Data Factory to Bring SAP Data to Cloud (Round 2)

/

Since our last blog posted 2 years ago, we received multiple requests to perform a comparison with Azure Data Factory’s “SAP Table” connector, in the context of bringing SAP data to cloud. We did some tests and would like to share our finding.

One-Line Summary

AecorSoft Table connector through ADF SSIS IR is multiple times faster and significantly more scalable than ADF SAP Table connector.

Test Setup

SAP Source: IDES EhP7 with SQL Server 2012R2

OS: Windows Server 2012R2 Hyper-V VM

Hardware: 16GB RAM in VM, 16 v-core of Xeon E5-2670 v1 2.6GHz

ADF SSIS IR: Standard_D2_v3 2 Core(s), 8192MB, 1 node

Destination: Azure Blob

Test Case 1 - BKPF

112 columns, 584,448 rows

AecorSoft and ADF SSIS IR: 1m 34s

ADF SAP Table via Selfhosted IR: 4m 44s

Quite note: the network to Azure is a bottleneck in our test setup. On this old workstation, if the destination is local CSV file, AecorSoft would only require around 23 seconds to complete. Therefore, about 1 minute is spent on the data transfer to Azure.

As the network limitation applies to both scenarios, we would live with this result for now. If such bottleneck is eliminated, the performance difference would be much bigger.

Test Case 2 - BSEG

354 columns, 1,678,379 rows

AecorSoft and ADF SSIS IR: 17m 7s

ADF SAP Table via Selfhosted IR: FAILED. It caused page allocation error short dump on SAP!

ADF SAP shortdump.png

Looking at the statistics, the network bandwidth issue is still very concerning. For those customers who have Express Route, the bottleneck should be much mitigated.

Conclusion

First, we are grateful that both Microsoft and SAP have created very healthy and sustainable ecosystems for partners and customers. In return, as one of the experts in SAP world, AecorSoft team also shared invaluable knowledge to senior leaders from these software giants and other partners, without holding back. Through everyone’s innovation, collaboration and healthy competition, customers can benefit the most.

Now focusing on the particular SAP Table extraction scenario alone, we feel that performing the ad-hoc queries through the native ADF SAP Table connector certainly has its value. But if performance, scalability, security, quality, delta capability, productivity, stress on SAP, and network bandwidth utilization matter to you, the SAP Premium Certified AecorSoft software is the obvious long-term choice for enterprises.