The Pursuit of SAP ETL Greatness - New Record of SAP Data Extraction
/Preface
Among the variety of metrics, the data extraction performance is one of the most critical to consider when SAP customers are dealing with the massive amount of SAP data. Through the continuous innovations, AecorSoft is excited to show off a new record of SAP data extraction performance!
This article does not intend to cover comprehensive test cases and tool comparisons which were covered in older posts already. The objective is to compare with our own legacy and let the newest fact and data tell the story. Through the software and hardware improvements, sky is the limit!
Test Setup
Source Object
BKPF on SAP IDES EhP7 (584K rows; 112 columns), extracted through SAP Application Layer
Why BKPF? There are a few reasons. First, it is one of the most critical transactional tables in SAP that every SAP customer is super familiar with. Second, it is a transparent table which does not have the additional operational overhead from clustered tables. Third, it is a table we have been using as a test object in most of our previous technical articles. Fourth, its schema nicely consists of 100+ columns with different data types.
Hardware
Commodity Workstation with AMD Ryzen 5 3600 CPU (6 cores)
Why AMD Ryzen? It is the “best-bang-for-the-buck” option on the market right now. The CPU alone is below USD 200 nowadays, yet its CPU performance exceeds other competitors in its price range or even higher. Isn’t that the same spirit of AecorSoft?
The workstation has 32GB DDR4-3200 RAM; 256GB SATA SSD for OS; 1TB SATA SSD for Hyper-V images.
OS Platforms
Host OS: Windows Server 2019 with HyperV
Guest OS1: Windows Server 2012R2 (for SAP IDES EhP7 with AecorSoft Turbo Function installed); 16GB RAM and 4 (out of the 6) CPU cores allocated
Guest OS2: Windows Server 2012R2 (for SSIS 2017 and AecorSoft Integration Service for SSIS); 8GB RAM; 4 (out of the 6) CPU cores allocated
Test Case
Use SSIS to extract entire BKPF table and write into local CSV file
Result
The job completes under 17 seconds! Some readers may still recall that our previous tests on older computers with Xeon and i7 CPUs and the previous Turbo function yielded the the result around 24 - 27 seconds. With better CPU and latest AecorSoft Turbo Function, a new record is born.
The output file size is 438MB (UTF-16 encoding).
With both SAP and SSIS running in VMs on same bare metal workstation, the host OS CPU usage is only peaked at 26-30%:
BSEG Challenge
Another test was performed against BSEG (345 columns) on same platform:
The output file is nearly 3GB:
1.68 millions of BSEG records were extracted to 2.9GB CSV file under 2m 42s, through SAP application layer!
Thoughts
438MB of SAP BKPF data extraction under 17 seconds. It is not magic. It is not rocket science. It is the beauty of Computer Science algorithms with a lot of hard work to make it happen.
AecorSoft Turbo Function makes a huge difference here. After all, it is the ONLY SAP data extraction solution in the world which is premium-certified by SAP.
The newer generation of commodity-grade CPU also contributes nicely to the result. The immediate takeaway for our readers is that when you provision VM systems through your favorite cloud vendors, go with the newer generation of CPUs whenever possible,
