Performance Evaluation v1

Report sponsored by Google

Database performance is crucial, directly impacting the user experience with applications and AI agents, as well as business operations and data integrity. Fast transaction processing, low latency, and high throughput enable businesses to operate efficiently, drive revenue, and maintain competitiveness. Conversely, poor performance leads to frustrated users, reduced productivity, and increased risk to the business. 

Prioritizing performance ensures seamless transactions, efficient data processing, and reliable operations. It maintains organizational credibility, prevents maintenance complexities, and safeguards against security threats. By investing in high-performance databases, organizations can drive growth, improve user satisfaction, and stay competitive in their market.

Investing in the most suitable database—with performance as a key consideration—ensures applications are running smoothly and securely, ultimately leading to better overall business outcomes. 

Google’s AlloyDB database is a fully managed database service offered by Google Cloud, designed for PostgreSQL workloads. It's built to provide high performance, scalability, and reliability for enterprise applications.

We tested AlloyDB against AWS Aurora for PostgreSQL to determine the comparative performance and price-performance. We used TPC-C, which is a widely recognized benchmark for measuring database performance. 

AlloyDB outperforms Aurora in the TPC-C benchmark, achieving a significantly higher transactions per minute (TPM) score of 2,875,803 compared to Aurora's 1,245,459¹. This indicates that AlloyDB can handle a higher volume of transactions, making it more suitable for workloads at all scales.

 AlloyDB's superior performance is complemented by its better price-performance ratio of $0.075 per transaction, compared to Aurora's $0.182. This translates to AlloyDB being 2.42 times more cost-effective than Aurora. With lower hourly, annual, and three-year costs, AlloyDB offers a more efficient and cost-effective solution for organizations with high-transactional workloads.

These tests highlight AlloyDB’s compelling value proposition, making it a prime choice for enterprises seeking great performance and great value with a transactional database.  

¹ _{When the database fits completely into the buffer cache. This approach minimizes the impact of disk I/O variability, allowing us to isolate the database's ability to process transactions. We also show testing with a partial database in cache, which yields even more dramatic results for AlloyDB.}

This section analyzes the methods we used in our database transactional performance testing. The benchmark was executed using the following setup, environment, standards, and configurations:

HammerDB TPROC-C Workload

The workload and data used in the benchmark were a workload derived from the well-recognized industry standard benchmark TPC-C. From tpc.org²:

“TPC-C is an on-line transaction processing (OLTP) benchmark. TPC-C involves a mix of five concurrent transactions of different types and complexity either executed on-line or queued for deferred execution. The database is comprised of nine types of tables with a wide range of record and population sizes. … While the benchmark portrays the activity of a wholesale supplier, TPC-C is not limited to the activity of any particular business segment, but, rather represents any industry that must manage, sell, or distribute a product or service.”

However, this is NOT an official TPC-C and the results are not comparable with other published TPC-C results. To perform a TPC-C-like workload, most people use a driver that is already built. There are several TPC-C-like drivers available for free. We used HammerDB TPROC-C v.4.6³.  

Table 1. HammerDB TPROC-C Testing Parameters

This TPC-C workload simulation tests database performance under demanding conditions, featuring high transaction volumes with 1,728 and 9,600 warehouses, multiple user connections with 672 virtual users, and standardized testing procedures with a 10-minute ramp-up period and 60-minute measurement period.

Infrastructure

To perform our testing, we compared the performance of:

• Google Cloud AlloyDB for PostgreSQL

• Amazon Web Services Aurora for PostgreSQL

Table 2. Deployed Systems

Test Measurements

Our benchmark collected two measurements.

• TPM (transactions per minute): Measures the throughput of database transactions that a system can handle per minute, reflecting its transaction processing capability.

• Price-performance: Measures the cost-effectiveness of a system by calculating the total price per transaction per minute (TPM), indicating how efficiently the system performs transactions relative to its price.

TPM and price-performance are pivotal measurements for evaluating transactional databases. TPM assesses transaction throughput, scalability, and real-world applicability by measuring the database's ability to process transactions efficiently, handle concurrent requests, and simulate real-world workloads. Price-performance evaluates cost-effectiveness, comparing performance to costs, enabling value assessments across databases and optimizing resource allocation.

To calculate the price-performance, we used the following formula:

Price-Performance ($) = Total System Cost for 3 Years ($) ÷ Transactions Per Minute

These measurements are chosen for their relevance to transactional use cases, facilitating comparability across databases and informing optimization, scaling, and cost management decisions. By considering TPM and price-performance, users gain comprehensive insights into database performance, scalability, and cost-effectiveness, guiding informed decisions for optimal database selection and configuration.

² _{More can be learned about the TPC-C benchmark at http://www.tpc.org/tpcc/.}
³ _{For more information about or to download HammerDB, visit https://www.hammerdb.com/.} 

The benchmark test compares the performance and cost of AlloyDB and RDS Aurora, both running PostgreSQL v16.3. 

Table 3. Database Benchmark Performance Metrics

Here are the key quantitative results:

Instance Costs

• AlloyDB: $8.220/hour, $72,007/year, $216,022/3 years

• Aurora: $8.616/hour, $75,476/year, $226,428/3 years

Performance Metrics

• AlloyDB: 2,875,803 Best TPM (transactions per minute) @ 672 virtual users and 1,728 warehouses

• Aurora: 1,245,459 Best TPM at 672 virtual users and 1,728 warehouses

• Performance difference: AlloyDB is 131% (over 2x) faster than Aurora (calculated as (2,875,803 - 1,245,459) / 1,245,459)

Price-Performance Ratio

• AlloyDB: $0.075 per transaction

• Aurora: $0.182 per transaction

• Price-performance difference: AlloyDB is 2.42 times more cost-effective than Aurora (calculated as $0.182 / $0.075)

In evaluating database systems, price-performance is often the most important metric, as it reflects the system's ability to deliver high performance at a reasonable cost. This metric helps businesses make informed decisions about their database investments, balancing performance requirements with budget constraints. By prioritizing price-performance, businesses can optimize their database infrastructure for both efficiency and cost-effectiveness.

Figure 1. Price-Performance Per Transaction

As Figure 1 shows, despite the modestly lower instance costs of AlloyDB, the high performance advantage of AlloyDB creates a significant price-performance advantage for AlloyDB.

Figure 2. Best Throughput Achieved with Database Fitting into Buffer Cache

As seen in Figure 2, when the database fits into the buffer cache, AlloyDB achieves 2,875,803 TPM, which is 130% higher than AWS Aurora's 1,245,459 TPM. Figure 3 shows that when the database partially fits into the buffer cache, AlloyDB's throughput decreases to 1,799,039 TPM, but it still outperforms AWS Aurora by 215%.

Aurora's throughput is significantly lower than AlloyDB's in both scenarios. In the first scenario (Figure 2), Aurora achieves 1,245,459 TPM, and in the second (Figure 3), it achieves 571,981 TPM.

Figure 3. Peak CPU Utilization with Database Fitting into Buffer Cache

Figure 4. Peak CPU Utilization with Database Fitting into Buffer Cache

Figure 5. Peak CPU Utilization with Database Partially Fitting into Buffer Cache

Figure 4 shows that when the database fits into the buffer cache, AlloyDB's peak CPU utilization is 91.0%, indicating that the database is efficiently utilizing available resources. When the database partially fits into the buffer cache, as seen in Figure 5, CPU utilization increases to 96.6%, which may indicate that the database is approaching its resource limits.

Aurora's peak CPU utilization is lower than AlloyDB's in both scenarios. In the first scenario (Figure 4), Aurora's CPU utilization is 77.8%, and in the second (Figure 5), it's 51.1%. CPU resources not being fully utilized may indicate that the system is not optimized for the workload, potentially leading to suboptimal performance. Lower CPU utilization might indicate bottlenecks in other areas, such as disk I/O, network, or database locking, which could be limiting overall system performance.

AlloyDB delivers superior performance in the TPC-C benchmark, with a TPM score of 2,875,803, outpacing Aurora's 1,245,459. These performance and price-performance advantages make AlloyDB suited for transactional workloads.

 AlloyDB's strong performance is matched by its cost-effectiveness, with a price-performance ratio of $0.075 per transaction, significantly outperforming Aurora's $0.182. This makes AlloyDB 2.42 times more cost-effective, offering substantial savings with lower hourly, annual, and three-year costs.

The significant performance and cost differences between AlloyDB and Aurora have important implications for businesses. For organizations with high transactional workloads (or those that aspire to high transactional workloads), AlloyDB's superior performance will lead to improved user experience, increased productivity, and enhanced competitiveness. Additionally, the cost savings associated with using AlloyDB could be substantially more than in our tests, particularly for large-scale deployments. Moreover, customers migrating off of Amazon Aurora to AlloyDB may need a smaller CPU footprint for the same workload (due to the superior performance of AlloyDB), and further reduce cost.

The results of this benchmark test can inform cloud database strategy for organizations. When evaluating cloud database options, consider not only the raw performance of the database but also the price-performance ratio. AlloyDB's strong showing in this benchmark suggests that it is a compelling option for high-performance and cost-effective database solutions. While this report focused on transactional workloads, AlloyDB also excels for analytical/HTAP workloads and AI workloads (vector processing).

As cloud database technology continues to evolve, it's essential to stay informed about the latest developments and advancements. The results of this benchmark test highlight the importance of ongoing evaluation and assessment of cloud database options. By staying up to date with the latest performance and cost metrics, users can make informed decisions about their cloud database strategy and optimize their investments for maximum return.

Performance is important, but it is only one criterion for a PostgreSQL database selection. This test is a point-in-time check into specific performance. There are numerous other factors to consider in selection across administration, features and functionality, workload management, user interface, scalability, vendor, reliability, and numerous other criteria. It is also our experience that performance changes over time. Moreover, a performance leader can reach a point of diminishing returns and viable contenders can close the gap quickly.

GigaOm runs all its performance tests to strict ethical standards. The results of the report are the objective results of the application of load tests to the simulations described in the report. The report clearly defines the selected criteria and process used to establish the field test. The report also clearly describes the tools and workloads used. The reader is left to determine for themselves how to qualify the information for their individual needs. The report does not make any claim regarding the third-party certification and presents the objective results received from the application of the process to the criteria as described in the report. The report strictly measures performance and does not purport to evaluate other factors that potential customers may find relevant when making a purchase decision.

This is a sponsored report. Google chose the competitors and the test. GigaOm chose the most compatible configurations and ran the testing workloads. Choosing compatible configurations is subject to judgment. We have attempted to describe our decisions fully in this report.

Google Cloud offers a range of database solutions, including AlloyDB, Bigtable, and Firestore. AlloyDB is a fully-managed PostgreSQL-compatible database service that provides high performance, scalability, and security for enterprise databases. With AlloyDB, businesses can modernize their database infrastructure and improve application performance.

William McKnight is a former Fortune 50 technology executive and database engineer. An Ernst & Young Entrepreneur of the Year finalist and frequent best practices judge, he helps enterprise clients with action plans, architectures, strategies, and technology tools to manage information.

Currently, William is an analyst for GigaOm Research who takes corporate information and turns it into a bottom-line-enhancing asset. He has worked with Dong Energy, France Telecom, Pfizer, Samba Bank, ScotiaBank, Teva Pharmaceuticals, and Verizon, among many others. William focuses on delivering business value and solving business problems utilizing proven approaches in information management.

Jake Dolezal is a contributing analyst at GigaOm. He has two decades of experience in the information management field, with expertise in analytics, data warehousing, master data management, data governance, business intelligence, statistics, data modeling and integration, and visualization. Jake has solved technical problems across a broad range of industries, including healthcare, education, government, manufacturing, engineering, hospitality, and restaurants. He has a doctorate in information management from Syracuse University.

GigaOm provides technical, operational, and business advice for IT’s strategic digital enterprise and business initiatives. Enterprise business leaders, CIOs, and technology organizations partner with GigaOm for practical, actionable, strategic, and visionary advice for modernizing and transforming their business. GigaOm’s advice empowers enterprises to successfully compete in an increasingly complicated business atmosphere that requires a solid understanding of constantly changing customer demands.

GigaOm works directly with enterprises both inside and outside of the IT organization to apply proven research and methodologies designed to avoid pitfalls and roadblocks while balancing risk and innovation. Research methodologies include but are not limited to adoption and benchmarking surveys, use cases, interviews, ROI/TCO, market landscapes, strategic trends, and technical benchmarks. Our analysts possess 20+ years of experience advising a spectrum of clients from early adopters to mainstream enterprises.

GigaOm’s perspective is that of the unbiased enterprise practitioner. Through this perspective, GigaOm connects with engaged and loyal subscribers on a deep and meaningful level.

© Knowingly, Inc. 2025 "Performance Evaluation" is a trademark of Knowingly, Inc. For permission to reproduce this report, please contact sales@gigaom.com.