Introduction

This is a report for the insert benchmark with 160M docs and 8 client(s). It is generated by scripts (bash, awk, sed) and Tufte might not be impressed. An overview of the insert benchmark is here and a short update is here. Below, by DBMS, I mean DBMS+version.config. An example is my8020.c10b40 where my means MySQL, 8020 is version 8.0.20 and c10b40 is the name for the configuration file.

The test server is a c2-standard-30 with 15 cores, hyperthreads disabled, 120G RAM and XFS with SW RAID 0 across 4 NVMe SSD. The benchmark was run with 8 clients and there were 1 or 3 connections per client (1 for queries or inserts without rate limits, 1+1 for rate limited inserts+deletes). It uses 8 tables, 1 per client. It loads 20M rows per table without secondary indexes, creates secondary indexes, then inserts 200M rows per table with a delete per insert to avoid growing the table. It then does 3 read+write tests for 3600s each that do queries as fast as possible with 100, 500 and then 1000 inserts/second/client concurrent with the queries. There are also deletes to match the inserts and avoid growing the table. The database is cached by Postgres. Clients and the DBMS share one server. The per-database configs are in the per-database subdirectories here.

The tested DBMS are:


Contents


Summary

The numbers are inserts/s for l.i0 and l.i1, indexed docs (or rows) /s for l.x and queries/s for q*.2. The values are the average rate over the entire test for inserts (IPS) and queries (QPS). The range of values for IPS and QPS is split into 3 parts: bottom 25%, middle 50%, top 25%. Values in the bottom 25% have a red background, values in the top 25% have a green background and values in the middle have no color. A gray background is used for values that can be ignored because the DBMS did not sustain the target insert rate. Red backgrounds are not used when the minimum value is within 80% of the max value.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s30 10126582320290244200114462111191109159
pg154_o3_native_lto.cx7a27_gcp_c2s30 10062892354412232660114165110971109719
pg16beta1_o3_native_lto.cx7a27_gcp_c2s30 10000002354412250078113747111179108648
pg16beta2_o3_native_lto.cx7a27_gcp_c2s30 9876542354412226244113646110352108196
pg16beta3_o3_native_lto.cx7a27_gcp_c2s30 10000002389552233816113309109984109044

This table has relative throughput, throughput for the DBMS relative to the DBMS in the first line, using the absolute throughput from the previous table. Values less than 0.95 have a yellow background. Values greater than 1.05 have a blue background.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s30 1.001.001.001.001.001.00
pg154_o3_native_lto.cx7a27_gcp_c2s30 0.991.010.951.001.001.01
pg16beta1_o3_native_lto.cx7a27_gcp_c2s30 0.991.011.020.991.001.00
pg16beta2_o3_native_lto.cx7a27_gcp_c2s30 0.981.010.930.990.990.99
pg16beta3_o3_native_lto.cx7a27_gcp_c2s30 0.991.030.960.990.991.00

This lists the average rate of inserts/s for the tests that do inserts concurrent with queries. For such tests the query rate is listed in the table above. The read+write tests are setup so that the insert rate should match the target rate every second. Cells that are not at least 95% of the target have a red background to indicate a failure to satisfy the target.

dbmsq100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s3079839877978
pg154_o3_native_lto.cx7a27_gcp_c2s3079839877978
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3079739877976
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3079739877978
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3079739877978
target80040008000

l.i0

l.i0: load without secondary indexes. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3097.9781.9620.0500.0010.0070.0030.292
pg154_o3_native_lto.cx7a27_gcp_c2s3098.0481.8970.0440.0010.0080.0020.351
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3098.0751.8680.0460.0020.0060.0030.368
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3097.3912.5570.0420.0010.0070.0030.308
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3098.1071.8400.0430.0010.0080.0030.337

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1012658	0	0	0.0	997.0	332.9	0.000	0.000	0.001	0.337	85486	77.6	0.084	11	15.3	41.4	5.4	0.292	143445	43268	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1006289	0	0	0.0	996.7	335.4	0.000	0.000	0.001	0.341	85371	77.7	0.085	12	15.3	41.4	1.7	0.351	143544	47947	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1000000	0	0	0.0	973.0	328.5	0.000	0.000	0.001	0.336	83207	76.4	0.083	11	15.3	41.4	1.7	0.368	142992	53142	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
987654	0	0	0.0	970.0	325.6	0.000	0.000	0.001	0.338	84170	76.3	0.085	12	15.3	41.4	1.7	0.308	140748	51743	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1000000	0	0	0.0	993.2	332.7	0.000	0.000	0.001	0.341	84247	77.6	0.084	12	15.3	41.4	4.5	0.337	142893	47548	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

l.x: create secondary indexes.

Average throughput:

Image

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
2320290	0	0	0.0	893.6	403.2	0.000	0.000	0.000	0.178	8089	41.7	0.003	3	29.4	63.3	13.2	0.002	NA	NA	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	901.0	405.2	0.000	0.000	0.000	0.176	8467	41.3	0.004	3	29.4	63.3	13.2	0.002	NA	NA	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	939.4	415.0	0.000	0.000	0.000	0.180	8828	41.0	0.004	3	29.4	63.4	13.1	0.002	NA	NA	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	949.2	420.9	0.000	0.000	0.000	0.183	8575	40.9	0.004	3	29.4	63.4	13.1	0.002	NA	NA	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
2389552	0	0	0.0	946.8	419.7	0.000	0.000	0.000	0.180	9041	41.0	0.004	3	29.4	63.4	13.2	0.002	NA	NA	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s300.04599.7330.1600.0590.0040.167
pg154_o3_native_lto.cx7a27_gcp_c2s300.08899.6940.1630.0520.003nonzero0.274
pg16beta1_o3_native_lto.cx7a27_gcp_c2s300.19799.5770.1580.0640.0040.226
pg16beta2_o3_native_lto.cx7a27_gcp_c2s300.27899.4730.1880.0570.0030.176
pg16beta3_o3_native_lto.cx7a27_gcp_c2s300.18499.5910.1660.0560.0030.173

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3078.5428.04211.3692.0280.0180.0020.173
pg154_o3_native_lto.cx7a27_gcp_c2s3078.9927.28710.8012.8980.0200.002nonzero0.274
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3077.2408.93812.2931.5070.0190.0020.223
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3077.1688.36711.6062.8390.0180.0020.173
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3078.3917.90210.8892.7970.0200.0020.167

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
244200	0	0	0.0	1174.9	192.3	0.000	0.000	0.005	0.807	95317	78.3	0.390	48	41.8	111.9	30.8	0.167	35584	6992	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
232660	0	0	0.0	1106.8	180.6	0.000	0.000	0.005	0.795	90417	77.4	0.389	50	41.6	111.7	7.2	0.274	35911	6293	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
250078	0	0	0.0	1181.6	196.2	0.000	0.000	0.005	0.803	97518	78.7	0.390	47	41.8	111.8	24.8	0.226	35967	7092	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
226244	0	0	0.0	1088.9	175.5	0.000	0.000	0.005	0.795	91084	74.6	0.403	49	41.7	111.7	37.8	0.176	34612	6143	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
233816	0	0	0.0	1102.7	181.8	0.000	0.000	0.005	0.796	91657	76.5	0.392	49	41.6	111.7	25.8	0.173	35922	4995	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.9960.0030.001nonzero0.010
pg154_o3_native_lto.cx7a27_gcp_c2s3099.9970.0030.001nonzero0.005
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.9970.0030.001nonzero0.009
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.9960.0030.001nonzero0.005
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.9960.0030.001nonzero0.005

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.8400.1600.007
pg154_o3_native_lto.cx7a27_gcp_c2s3099.8990.1010.013
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.8840.1160.013
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.7000.3000.008
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.7990.2010.009

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3053.47623.04323.4580.0230.005
pg154_o3_native_lto.cx7a27_gcp_c2s3064.70319.09213.0893.1160.012
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3055.68818.92724.9690.4170.009
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3061.27117.43418.9322.3630.010
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3057.26021.98620.6890.0640.012

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
798	114462	0	0.0	181.4	9.5	0.000	0.000	0.227	12.249	435556	52.9	3.805	69	41.8	111.9	12.4	0.010	14262	14017	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
798	114165	0	0.0	173.7	9.6	0.000	0.000	0.218	12.337	434521	52.7	3.806	69	41.5	111.6	10.4	0.005	14240	13937	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
797	113747	0	0.0	175.1	9.5	0.000	0.000	0.220	12.180	433046	52.9	3.807	70	41.8	111.8	1.6	0.009	14240	13937	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
797	113646	0	0.0	195.7	10.0	0.000	0.000	0.246	12.792	432649	52.9	3.807	70	41.7	111.7	1.7	0.005	14086	13745	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
797	113309	0	0.0	204.8	9.7	0.000	0.000	0.257	12.396	431321	52.8	3.807	70	41.6	111.7	7.4	0.005	14240	13974	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.9860.0090.005nonzerononzerononzero0.068
pg154_o3_native_lto.cx7a27_gcp_c2s3099.9860.0090.005nonzerononzerononzero0.070
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.9850.0100.005nonzerononzero0.018
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.9860.0090.005nonzero0.015
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.9870.0090.005nonzerononzero0.018

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.7970.2020.0010.018
pg154_o3_native_lto.cx7a27_gcp_c2s3099.8090.1880.0030.023
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.6610.3330.0060.021
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.8360.1620.0020.020
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.8360.1630.0020.019

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3077.93910.2647.2564.5410.0010.019
pg154_o3_native_lto.cx7a27_gcp_c2s3077.2995.5006.81110.3870.0030.025
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3076.8416.61710.2616.2720.0080.021
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3076.7095.9697.8219.5000.0010.019
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3078.2039.0425.0697.6840.0030.031

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3987	111191	0	0.0	280.8	14.7	0.000	0.000	0.070	3.768	423484	53.8	3.809	73	41.8	106.5	13.7	0.068	13953	11747	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
3987	110971	0	0.0	277.1	14.4	0.000	0.000	0.070	3.708	422473	53.9	3.807	73	41.5	106.0	14.0	0.070	14001	11859	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
3987	111179	0	0.0	285.4	14.3	0.000	0.000	0.072	3.665	423185	53.8	3.806	73	41.8	106.3	26.8	0.018	13985	11891	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
3987	110352	0	0.0	270.8	14.6	0.000	0.000	0.068	3.746	420227	54.0	3.808	73	41.7	106.1	11.5	0.015	13718	11411	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
3987	109984	0	0.0	270.4	14.8	0.000	0.000	0.068	3.813	418883	53.9	3.809	74	41.6	106.0	12.8	0.018	13745	11555	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.9580.0320.009nonzerononzerononzero0.091
pg154_o3_native_lto.cx7a27_gcp_c2s3099.9600.0300.009nonzerononzero0.026
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.9620.0300.009nonzerononzero0.025
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.9550.0350.010nonzerononzero0.025
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.9530.0360.010nonzerononzerononzero0.087

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.4610.5230.015nonzero0.072
pg154_o3_native_lto.cx7a27_gcp_c2s3099.2910.6840.0260.035
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.6520.3370.0110.033
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.3970.5830.0200.028
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.3200.6600.020nonzero0.080

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3078.6387.3806.3527.6160.014nonzero0.070
pg154_o3_native_lto.cx7a27_gcp_c2s3076.9445.0557.9949.9920.0150.032
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3077.8766.2518.3157.5530.0060.033
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3075.9476.6198.2399.1890.0050.026
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3076.7678.0756.6698.4830.0050.023

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7978	109159	0	0.0	181.4	16.7	0.000	0.000	0.023	2.146	413001	55.0	3.783	76	41.6	99.3	24.9	0.091	13777	11811	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
7978	109719	0	0.0	178.5	16.7	0.000	0.000	0.022	2.142	414915	55.1	3.782	75	41.5	99.1	26.2	0.026	13728	11715	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
7976	108648	0	0.0	184.0	16.7	0.000	0.000	0.023	2.148	412183	55.0	3.794	76	41.8	99.4	24.4	0.025	13649	11555	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
7978	108196	0	0.0	177.2	16.6	0.000	0.000	0.022	2.136	409940	55.1	3.789	76	41.7	99.2	14.0	0.025	13377	11352	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
7978	109044	0	0.0	179.6	16.6	0.000	0.000	0.023	2.136	412608	55.0	3.784	76	41.6	99.0	13.7	0.087	13617	11683	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i0

l.i0: load without secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1012658	0	0	0.0	997.0	332.9	0.000	0.000	0.001	0.337	85486	77.6	0.084	11	15.3	41.4	5.4	0.292	143445	43268	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1006289	0	0	0.0	996.7	335.4	0.000	0.000	0.001	0.341	85371	77.7	0.085	12	15.3	41.4	1.7	0.351	143544	47947	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1000000	0	0	0.0	973.0	328.5	0.000	0.000	0.001	0.336	83207	76.4	0.083	11	15.3	41.4	1.7	0.368	142992	53142	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
987654	0	0	0.0	970.0	325.6	0.000	0.000	0.001	0.338	84170	76.3	0.085	12	15.3	41.4	1.7	0.308	140748	51743	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1000000	0	0	0.0	993.2	332.7	0.000	0.000	0.001	0.341	84247	77.6	0.084	12	15.3	41.4	4.5	0.337	142893	47548	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

l.x: create secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
2320290	0	0	0.0	893.6	403.2	0.000	0.000	0.000	0.178	8089	41.7	0.003	3	29.4	63.3	13.2	0.002	NA	NA	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	901.0	405.2	0.000	0.000	0.000	0.176	8467	41.3	0.004	3	29.4	63.3	13.2	0.002	NA	NA	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	939.4	415.0	0.000	0.000	0.000	0.180	8828	41.0	0.004	3	29.4	63.4	13.1	0.002	NA	NA	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
2354412	0	0	0.0	949.2	420.9	0.000	0.000	0.000	0.183	8575	40.9	0.004	3	29.4	63.4	13.1	0.002	NA	NA	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
2389552	0	0	0.0	946.8	419.7	0.000	0.000	0.000	0.180	9041	41.0	0.004	3	29.4	63.4	13.2	0.002	NA	NA	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
244200	0	0	0.0	1174.9	192.3	0.000	0.000	0.005	0.807	95317	78.3	0.390	48	41.8	111.9	30.8	0.167	35584	6992	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
232660	0	0	0.0	1106.8	180.6	0.000	0.000	0.005	0.795	90417	77.4	0.389	50	41.6	111.7	7.2	0.274	35911	6293	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
250078	0	0	0.0	1181.6	196.2	0.000	0.000	0.005	0.803	97518	78.7	0.390	47	41.8	111.8	24.8	0.226	35967	7092	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
226244	0	0	0.0	1088.9	175.5	0.000	0.000	0.005	0.795	91084	74.6	0.403	49	41.7	111.7	37.8	0.176	34612	6143	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
233816	0	0	0.0	1102.7	181.8	0.000	0.000	0.005	0.796	91657	76.5	0.392	49	41.6	111.7	25.8	0.173	35922	4995	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
798	114462	0	0.0	181.4	9.5	0.000	0.000	0.227	12.249	435556	52.9	3.805	69	41.8	111.9	12.4	0.010	14262	14017	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
798	114165	0	0.0	173.7	9.6	0.000	0.000	0.218	12.337	434521	52.7	3.806	69	41.5	111.6	10.4	0.005	14240	13937	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
797	113747	0	0.0	175.1	9.5	0.000	0.000	0.220	12.180	433046	52.9	3.807	70	41.8	111.8	1.6	0.009	14240	13937	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
797	113646	0	0.0	195.7	10.0	0.000	0.000	0.246	12.792	432649	52.9	3.807	70	41.7	111.7	1.7	0.005	14086	13745	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
797	113309	0	0.0	204.8	9.7	0.000	0.000	0.257	12.396	431321	52.8	3.807	70	41.6	111.7	7.4	0.005	14240	13974	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3987	111191	0	0.0	280.8	14.7	0.000	0.000	0.070	3.768	423484	53.8	3.809	73	41.8	106.5	13.7	0.068	13953	11747	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
3987	110971	0	0.0	277.1	14.4	0.000	0.000	0.070	3.708	422473	53.9	3.807	73	41.5	106.0	14.0	0.070	14001	11859	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
3987	111179	0	0.0	285.4	14.3	0.000	0.000	0.072	3.665	423185	53.8	3.806	73	41.8	106.3	26.8	0.018	13985	11891	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
3987	110352	0	0.0	270.8	14.6	0.000	0.000	0.068	3.746	420227	54.0	3.808	73	41.7	106.1	11.5	0.015	13718	11411	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
3987	109984	0	0.0	270.4	14.8	0.000	0.000	0.068	3.813	418883	53.9	3.809	74	41.6	106.0	12.8	0.018	13745	11555	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7978	109159	0	0.0	181.4	16.7	0.000	0.000	0.023	2.146	413001	55.0	3.783	76	41.6	99.3	24.9	0.091	13777	11811	160m.pg153_o3_native_lto.cx7a27_gcp_c2s30
7978	109719	0	0.0	178.5	16.7	0.000	0.000	0.022	2.142	414915	55.1	3.782	75	41.5	99.1	26.2	0.026	13728	11715	160m.pg154_o3_native_lto.cx7a27_gcp_c2s30
7976	108648	0	0.0	184.0	16.7	0.000	0.000	0.023	2.148	412183	55.0	3.794	76	41.8	99.4	24.4	0.025	13649	11555	160m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
7978	108196	0	0.0	177.2	16.6	0.000	0.000	0.022	2.136	409940	55.1	3.789	76	41.7	99.2	14.0	0.025	13377	11352	160m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
7978	109044	0	0.0	179.6	16.6	0.000	0.000	0.023	2.136	412608	55.0	3.784	76	41.6	99.0	13.7	0.087	13617	11683	160m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i0

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	97.978	1.962	0.050	0.001	0.007	0.003	0.000	0.000	0.000	0.292	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	98.048	1.897	0.044	0.001	0.008	0.002	0.000	0.000	0.000	0.351	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	98.075	1.868	0.046	0.002	0.006	0.003	0.000	0.000	0.000	0.368	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	97.391	2.557	0.042	0.001	0.007	0.003	0.000	0.000	0.000	0.308	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	98.107	1.840	0.043	0.001	0.008	0.003	0.000	0.000	0.000	0.337	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

TODO - determine whether there is data for create index response time


l.i1

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.045	99.733	0.160	0.059	0.004	0.000	0.000	0.000	0.000	0.167	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.088	99.694	0.163	0.052	0.003	nonzero	0.000	0.000	0.000	0.274	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.197	99.577	0.158	0.064	0.004	0.000	0.000	0.000	0.000	0.226	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.278	99.473	0.188	0.057	0.003	0.000	0.000	0.000	0.000	0.176	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.184	99.591	0.166	0.056	0.003	0.000	0.000	0.000	0.000	0.173	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
78.542	8.042	11.369	2.028	0.018	0.002	0.000	0.000	0.000	0.000	0.173	pg153_o3_native_lto.cx7a27_gcp_c2s30
78.992	7.287	10.801	2.898	0.020	0.002	nonzero	0.000	0.000	0.000	0.274	pg154_o3_native_lto.cx7a27_gcp_c2s30
77.240	8.938	12.293	1.507	0.019	0.002	0.000	0.000	0.000	0.000	0.223	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
77.168	8.367	11.606	2.839	0.018	0.002	0.000	0.000	0.000	0.000	0.173	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
78.391	7.902	10.889	2.797	0.020	0.002	0.000	0.000	0.000	0.000	0.167	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.996	0.003	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.010	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.997	0.003	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.005	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.997	0.003	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.009	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.996	0.003	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.005	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.996	0.003	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.005	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	99.840	0.160	0.000	0.000	0.000	0.000	0.000	0.000	0.007	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.899	0.101	0.000	0.000	0.000	0.000	0.000	0.000	0.013	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.884	0.116	0.000	0.000	0.000	0.000	0.000	0.000	0.013	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.700	0.300	0.000	0.000	0.000	0.000	0.000	0.000	0.008	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.799	0.201	0.000	0.000	0.000	0.000	0.000	0.000	0.009	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
53.476	23.043	23.458	0.023	0.000	0.000	0.000	0.000	0.000	0.000	0.005	pg153_o3_native_lto.cx7a27_gcp_c2s30
64.703	19.092	13.089	3.116	0.000	0.000	0.000	0.000	0.000	0.000	0.012	pg154_o3_native_lto.cx7a27_gcp_c2s30
55.688	18.927	24.969	0.417	0.000	0.000	0.000	0.000	0.000	0.000	0.009	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
61.271	17.434	18.932	2.363	0.000	0.000	0.000	0.000	0.000	0.000	0.010	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
57.260	21.986	20.689	0.064	0.000	0.000	0.000	0.000	0.000	0.000	0.012	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.986	0.009	0.005	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.068	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.986	0.009	0.005	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.070	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.985	0.010	0.005	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.018	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.986	0.009	0.005	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.015	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.987	0.009	0.005	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.018	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	99.797	0.202	0.001	0.000	0.000	0.000	0.000	0.000	0.018	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.809	0.188	0.003	0.000	0.000	0.000	0.000	0.000	0.023	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.661	0.333	0.006	0.000	0.000	0.000	0.000	0.000	0.021	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.836	0.162	0.002	0.000	0.000	0.000	0.000	0.000	0.020	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.836	0.163	0.002	0.000	0.000	0.000	0.000	0.000	0.019	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
77.939	10.264	7.256	4.541	0.001	0.000	0.000	0.000	0.000	0.000	0.019	pg153_o3_native_lto.cx7a27_gcp_c2s30
77.299	5.500	6.811	10.387	0.003	0.000	0.000	0.000	0.000	0.000	0.025	pg154_o3_native_lto.cx7a27_gcp_c2s30
76.841	6.617	10.261	6.272	0.008	0.000	0.000	0.000	0.000	0.000	0.021	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
76.709	5.969	7.821	9.500	0.001	0.000	0.000	0.000	0.000	0.000	0.019	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
78.203	9.042	5.069	7.684	0.003	0.000	0.000	0.000	0.000	0.000	0.031	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.958	0.032	0.009	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.091	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.960	0.030	0.009	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.026	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.962	0.030	0.009	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.025	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.955	0.035	0.010	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.025	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.953	0.036	0.010	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.087	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	99.461	0.523	0.015	nonzero	0.000	0.000	0.000	0.000	0.072	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.291	0.684	0.026	0.000	0.000	0.000	0.000	0.000	0.035	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.652	0.337	0.011	0.000	0.000	0.000	0.000	0.000	0.033	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.397	0.583	0.020	0.000	0.000	0.000	0.000	0.000	0.028	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	99.320	0.660	0.020	nonzero	0.000	0.000	0.000	0.000	0.080	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
78.638	7.380	6.352	7.616	0.014	nonzero	0.000	0.000	0.000	0.000	0.070	pg153_o3_native_lto.cx7a27_gcp_c2s30
76.944	5.055	7.994	9.992	0.015	0.000	0.000	0.000	0.000	0.000	0.032	pg154_o3_native_lto.cx7a27_gcp_c2s30
77.876	6.251	8.315	7.553	0.006	0.000	0.000	0.000	0.000	0.000	0.033	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
75.947	6.619	8.239	9.189	0.005	0.000	0.000	0.000	0.000	0.000	0.026	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
76.767	8.075	6.669	8.483	0.005	0.000	0.000	0.000	0.000	0.000	0.023	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30