Parallelizing user-defined functions in the ETL workflow using orchestration style sheets - Ali, Syed Muhammad Fawad; Mey, Johannes; Thiele, Maik (Zielona Góra: Uniwersytet Zielonogórski)
Today`s ETL tools provide capabilities to develop custom code as user-defined functions (UDFs) to extend the expressiveness of the standard ETL operators. However, while this allows us to easily add new functionalities, it also comes with the risk that the custom code is not intended to be optimized, e.g., by parallelism, and for this reason, it performs poorly for data-intensive ETL workflows. In this paper we present a novel framework, which allows the ETL developer to choose a design pattern in order to write parallelizable code and generates a configuration for the UDFs to be executed in a distributed environment.
Keywords: ETL workflow; parallel ETL operators; parallel algorithmic skeletons; user-defined functions