Case Studies on the Effective Data Processing using In-Memory DBMS

InMemory DBMS and Disk DBMS Mechanism

Original Author: Oh, Sung Il (2012)

Advisor: Prof. Ahn, Hyun ChulKookmin

University Graduate School of Business IT

Abstract

The information level of enterprises is increasingly measured not by the quantity of data held, but by data processing speed and the ability to transform raw data into meaningful information. Among these factors, data processing speed has become the most critical element. The challenge of effectively processing massive amounts of data generated in short timeframes has emerged as one of the major information technology challenges facing today's enterprises.

While traditional large-volume data accumulated over time can be handled by conventional DBMS technologies, processing rapidly generated data requires specialized "high-speed data processing technology." This study examines In-Memory DBMS as an effective solution for rapid data processing and analyzes real-world implementation cases to verify its applicability and industrial effectiveness, particularly in telecommunications, finance, and manufacturing industries where high-speed data processing is critically required.

1. Introduction

1.1 Research Background and Objectives

Modern enterprises face unprecedented challenges in managing exponentially growing data volumes. Leading nations and companies are making continuous investments to secure competitive advantages in big data analytics, extracting valuable insights for personalized marketing, traffic management, tax administration, crime prevention, and disaster response.

From a technical perspective, providing enhanced data-driven services requires revolutionary approaches to data analysis and processing. In this context, In-Memory DBMS technology, which has existed for decades, is re-emerging as an effective alternative for data processing.

In-Memory DBMS history dates back to 1993 with research at Bell Labs. Despite rapid growth and widespread adoption across various fields, it still has limitations compared to traditional disk-based DBMS, particularly in storing and managing massive data volumes. However, In-Memory DBMS is finding its niche in specific applications where its role differs significantly from traditional disk-based databases.

This research examines the characteristics and functionality of In-Memory DBMS through in-depth analysis of implementation cases in telecommunications, finance, and manufacturing industries, evaluating whether In-Memory DBMS can substantially contribute to effective enterprise data processing.

2. In-Memory DBMS Overview

2.1 Fundamental Differences from Disk-Based DBMS

The fundamental difference between disk-based and In-Memory DBMS lies in data storage location. While disk DBMS stores tables and indexes on disk and loads specific data pages into memory when needed, In-Memory DBMS loads the entire database into memory at startup, with all record access occurring directly in memory.

Key Characteristics Comparison:

Feature	In-Memory DBMS	Disk-Based DBMS
Performance	High-speed data processing with minimal disk I/O bottlenecks	Limited by disk I/O operations
Data Capacity	Limited by physical memory size	Virtually unlimited storage capacity
Use Cases	OLTP systems, real-time processing	Large-scale data warehousing, historical data
Cost	Higher memory costs	Lower storage costs

2.2 Technical Advantages

Memory Management: Optimized data structures and memory pool management significantly improve performance compared to traditional memory allocation methods.

Concurrency Control: Multi-Version Concurrency Control (MVCC) eliminates conflicts between read and write operations, maximizing performance in high-user environments.

Query Optimization: Cost-based optimization algorithms specifically designed for memory address spaces minimize query processing time and improve data access speeds.

3. Case Study Results

Case 1: S Telecommunications - Integrated Billing System

Company Profile: Established 1984, 4,340 employees, leading telecommunications company with 8M 2G and 19.5M 3G mobile subscribers (2011).

System Challenges:

• IBM mainframe with DB2 unable to provide real-time billing services

• Limited flexibility for application integration and expansion

• High system maintenance and upgrade costs

Implementation Results:

• Real-time Processing: Billing information available within 5 minutes

• Transaction Volume: Increased from 150M to 4.5B calls per month

• System Efficiency: 50% resource optimization, enabling additional application deployment

• Architecture: Hot data (last 3 months) processed via Altibase In-Memory DBMS, cold data stored in Oracle

Performance Metrics:

Metric	Before Implementation	After Implementation
Processing Window	Batch (5 min intervals)	Real-time
Daily Transaction Volume	150M calls	Up to 400M calls
Monthly Call Analysis	Not available	4.5B calls
System Resource Utilization	At capacity	50% available resources

Case 2: S Securities - Futures & Options Trading System

Company Profile: Established 1982, 3,000 employees, major securities trading and brokerage firm with global operations.

System Challenges:

• Sybase DBMS experiencing persistent bottlenecks

• Inability to integrate order information, trader data, and transaction amounts

• Need for high availability and system redundancy

  
  

Implementation Results:

• Replaced primary Sybase DBMS with Altibase In-Memory DBMS

• 40GB memory allocation for hot data processing

• Implemented active-active replication for high availability

Performance Metrics:

Metric	Before Implementation	After Implementation	Improvement
Order Execution Time	80ms per order	3ms per order	26x faster
Order Processing Rate	750 orders/minute	20,000 orders/minute	26x increase
CPU Utilization	50-60%	10-20%	65% reduction

Case 3: L Company - APC Manufacturing System

Company Profile: Established 1985, 30,117 employees, second-largest electronics manufacturer in Korea, specializing in LCD display production.

System Challenges:

• Multiple separate DBMS instances for different production processes

• Complex system architecture with inefficient management

• Limited data processing performance preventing system integration

Implementation Results:

• Consolidated multiple production process databases into single DBMS

• Enabled real-time integrated production monitoring

• Scalable system architecture for future capacity expansion

Performance Metrics:

Metric	Before Implementation	After Implementation	Improvement
Insert Operations	5,800 TPS	50,000 TPS	9x increase
Select Operations	Not measurable	35,000 TPS	N/A
Move Operations	2.1B transactions/hour	4.2B+ transactions/hour	2x+ increase

4. Key Findings and Analysis

4.1 Performance Improvements

All three case studies demonstrated significant performance improvements:

• Processing Speed: 9-26x improvement in transaction processing times

• Throughput: Substantial increases in concurrent transaction handling

• System Efficiency: 50-65% reduction in resource utilization

• Real-time Capability: Transformation from batch to real-time processing

4.2 Business Impact

Cost Reduction: Consolidation of multiple systems and reduced maintenance costs

Operational Efficiency: Real-time data processing enabling faster business decisions

Scalability: Improved ability to handle growing data volumes and user loads

Competitive Advantage: Enhanced service quality and customer satisfaction

4.3 Technical Considerations

Memory Requirements: Substantial physical memory investments required

Data Volume Limitations: Not suitable for massive historical data storage

Hybrid Architecture: Optimal implementation combines In-Memory DBMS for hot data with traditional DBMS for cold data storage

5. Conclusions and Future Perspectives

This research demonstrates that In-Memory DBMS enables real-time high-speed transaction processing, significantly reducing data processing time and eliminating bottlenecks compared to traditional disk-based DBMS. The three industry case studies confirm substantial improvements in processing speed, system efficiency, and operational capabilities.

5.1 Key Advantages

• Dramatic improvement in data processing speed

• Elimination of disk I/O bottlenecks

• Enhanced real-time processing capabilities

• Reduced system resource utilization

  
  

5.2 Limitations and Considerations

• Memory storage constraints limiting large-scale data capacity

• Higher initial investment costs for memory infrastructure

• Not suitable for big data scenarios requiring massive historical data storage

  
  

5.3 Future Outlook

As memory prices continue to decline and processing requirements increase, In-Memory DBMS adoption is expected to accelerate across industries. The optimal future architecture likely involves hybrid systems combining In-Memory DBMS for real-time processing with traditional disk-based systems for long-term data storage.

The rapid generation of data in the modern era means that raw data has limited value and short lifecycles. However, when this data is processed in real-time and transformed into meaningful information, it gains significant preservation value. In-Memory DBMS technology, particularly when combined with traditional DBMS in hybrid architectures, will play a crucial role in this data processing ecosystem.

5.4 Recommendations

• Enterprise Decision Makers: Consider In-Memory DBMS for applications requiring real-time processing and high transaction volumes

• IT Professionals: Evaluate hybrid architectures combining In-Memory and disk-based systems

• Industry Practitioners: Focus on applications where processing speed is more critical than storage capacity

The study confirms that In-Memory DBMS represents a significant advancement in data processing technology, particularly suited for modern enterprises requiring rapid data analysis and real-time decision-making capabilities.

In-Memory_DBMS를_이용한_효과적인_데이터_처리_사례_연구.pdf

Download PDF • 760KB