Businesses nowadays get tremendously bigger. Dealing with large amounts of data becomes a challenge for the Data Warehouse. According to Gartner’s survey regarding data center infrastructure trends and challenges, “Data growth is the largest data center hardware and infrastructure challenge for large enterprises”.
In order to face these challenges many organizations found their relief in in-memory data platforms such as SAP HANA with its hybrid structure for both processing transactional workloads and analytical workloads completely in-memory. With the new release of SAP HANA, SAP introduces a new concept of Multi-Temperature and Data Lifecycle Management storing data efficiently on different types of storage devices based on data temperature, as shown in figure 1. Taking this into consideration, data which is accessed frequently (hot data) is stored in-memory on HANA while warm data can be stored on Extended Storage Host (Dynamic Tiering). Cold data is moved on less expensive storage devices outside the HANA Database only for the read purpose, known as Near-Line Storage (NLS, read more here).
Figure 1. Multi-Temperature Data Classification
One of our customer’s project goals was to move some of less frequented data on disk that still could be accessed and don’t keep it in HANA in-memory. In search for a suitable solution for our customer, who was looking to optimize HANA memory consumption and with this reducing license costs as well as improving the system stability, we have analyzed and investigate a possible SAP solution:
- Dynamic Tiering – unfortunately doesn’t fit our customer scenario due to additional license costs as well as many features and functions which are underdeveloped yet and will only be available in future releases.
Our first research analysis was focused on developing the Inspiricon HANA In-Memory Space Reduction project where we have evaluated different approaches such as:
- Migration of the existing data models to HANA optimized ones (LSA++).
- Identifying and deleting outdated data/objects that are not required for reporting any more.
- Defining with business and unloading historical data from HANA in-memory to HANA disk space.
- HANA DB compression.
The above listed approaches exploit SAP HANA features, e.g. partitioning of AdvancedDSO. We have developed a methodology, based on Inspiricon best practices, which does not imply additional license costs and does not increase the TCO (Total Costs of Ownership).
As a result, we have introduced the Data Lifecycle Management (DLM), which covers defining data keeping needs through Business/IT and methods to apply for a certain data model depending on its complexity and Business/IT requirements.
Figure 2. Inspiricon Methodology for DLM
Our project was based on a detailed system analysis identifying the biggest data objects that would be possible candidates for minimizing HANA memory consumption.
Figure 3. DLM approaches to reduce memory consumption
This solution helps to better organize the existing data model by either deleting data which is stored multiple times in different InfoProviders or eliminating BW objects in layers which become obsolete on HANA. As a consequence HANA in-memory is reduced.
This application was developed by Inspiricon in particular for this customer scenario. The main usage is focused on unloading certain objects’ tables from HANA in-memory to HANA disk space. Thus, HANA DB will automatically load them back to the memory, once data is accessed in any way.
This approach helps to decrease memory consumption by unloading just a number of predefined partitions excluding others, based typically on time characteristics like calendar year/ month or fiscal period. Partitioning is defined on BW and it is performed on the Database level and can be applied to the existing DSO (with restrictions) or to the AdvancedDSO depending on customer business requirements.
- As a result, the Inspiricon HANA In-Memory Space Reduction Project saved 20% of our customer HANA memory consumption without additional license costs and resources.
- The main deliverables were better loading performance and improvement of the system stability, smoother DW landscape – and with this decreasing considerably the annually SAP HANA memory licenses.
- Our consultancy added value is to deliver the best solution that matches our customer needs, implying our know-how, effective consulting and partnership.