Introduction

This is a report for the insert benchmark with 4000M docs and 8 client(s). It is generated by scripts (bash, awk, sed) and Tufte might not be impressed. An overview of the insert benchmark is here and a short update is here. Below, by DBMS, I mean DBMS+version.config. An example is my8020.c10b40 where my means MySQL, 8020 is version 8.0.20 and c10b40 is the name for the configuration file.

The test server is a c2-standard-30 with 15 cores, hyperthreads disabled, 120G RAM and XFS with SW RAID 0 across 4 NVMe SSD. The benchmark was run with 8 clients and there were 1 or 3 connections per client (1 for queries or inserts without rate limits, 1+1 for rate limited inserts+deletes). It uses 8 tables, 1 per client. It loads 500M rows per table without secondary indexes, creates secondary indexes, then inserts 40M rows per table with a delete per insert to avoid growing the table. It then does 3 read+write tests for 3600s each that do queries as fast as possible with 100, 500 and then 1000 inserts/second/client concurrent with the queries. There are also deletes to match the inserts and avoid growing the table. The database is larger than memory. Clients and the DBMS share one server. The per-database configs are in the per-database subdirectories here.

The tested DBMS are:


Contents


Summary

The numbers are inserts/s for l.i0 and l.i1, indexed docs (or rows) /s for l.x and queries/s for q*.2. The values are the average rate over the entire test for inserts (IPS) and queries (QPS). The range of values for IPS and QPS is split into 3 parts: bottom 25%, middle 50%, top 25%. Values in the bottom 25% have a red background, values in the top 25% have a green background and values in the middle have no color. A gray background is used for values that can be ignored because the DBMS did not sustain the target insert rate. Red backgrounds are not used when the minimum value is within 80% of the max value.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s30 11108021624736114261044679469974007
pg154_o3_native_lto.cx7a27_gcp_c2s30 11013221622759111031046039656495302
pg16beta1_o3_native_lto.cx7a27_gcp_c2s30 10914051739930101241046789567274696
pg16beta2_o3_native_lto.cx7a27_gcp_c2s30 1099505175520095641049909729977157
pg16beta3_o3_native_lto.cx7a27_gcp_c2s30 11095701734649112411036799421686676

This table has relative throughput, throughput for the DBMS relative to the DBMS in the first line, using the absolute throughput from the previous table. Values less than 0.95 have a yellow background. Values greater than 1.05 have a blue background.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s30 1.001.001.001.001.001.00
pg154_o3_native_lto.cx7a27_gcp_c2s30 0.991.000.971.001.021.29
pg16beta1_o3_native_lto.cx7a27_gcp_c2s30 0.981.070.891.001.011.01
pg16beta2_o3_native_lto.cx7a27_gcp_c2s30 0.991.080.841.011.031.04
pg16beta3_o3_native_lto.cx7a27_gcp_c2s30 1.001.070.980.990.991.17

This lists the average rate of inserts/s for the tests that do inserts concurrent with queries. For such tests the query rate is listed in the table above. The read+write tests are setup so that the insert rate should match the target rate every second. Cells that are not at least 95% of the target have a red background to indicate a failure to satisfy the target.

dbmsq100.1q500.1q1000.1
pg153_o3_native_lto.cx7a27_gcp_c2s3079839877811
pg154_o3_native_lto.cx7a27_gcp_c2s3079739887209
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3079739887822
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3079839877617
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3079739887486
target80040008000

l.i0

l.i0: load without secondary indexes. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3097.6432.3150.0290.0050.0080.0010.662
pg154_o3_native_lto.cx7a27_gcp_c2s3097.5502.4080.0280.0050.0090.0010.736
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3097.4142.5450.0270.0050.0080.0010.604
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3097.5932.3660.0290.0050.0070.0010.561
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3097.5192.4440.0270.0050.006nonzero0.803

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1110802	0	540	4.3	1385.9	253.5	0.000	0.004	0.001	0.234	92534	84.6	0.083	11	382.7	452.8	9.9	0.662	142424	81810	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1101322	0	550	4.3	1364.3	250.3	0.000	0.004	0.001	0.233	91830	84.3	0.083	11	382.7	452.8	17.8	0.736	141645	71622	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1091405	0	532	4.2	1367.4	249.4	0.000	0.004	0.001	0.234	91678	83.9	0.084	12	382.7	452.8	13.3	0.604	140346	70224	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
1099505	0	548	4.3	1375.2	251.8	0.000	0.004	0.001	0.235	91448	84.4	0.083	12	382.7	452.8	5.4	0.561	141144	75418	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1109570	0	549	4.3	1373.8	254.3	0.000	0.004	0.001	0.235	91674	84.8	0.083	11	382.7	452.8	9.2	0.803	141946	92935	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

l.x: create secondary indexes.

Average throughput:

Image

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1624736	0	3555	768.1	1244.9	277.6	0.002	0.484	0.001	0.175	13241	37.1	0.008	3	735.2	804.7	56.7	0.002	NA	NA	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1622759	0	3511	770.1	1243.1	280.0	0.002	0.486	0.001	0.177	13064	36.9	0.008	3	735.2	805.2	56.9	0.002	NA	NA	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1739930	0	2974	687.3	1330.4	300.3	0.002	0.404	0.001	0.177	18486	39.0	0.011	3	735.2	804.8	57.2	0.002	NA	NA	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
1755200	0	3008	697.7	1344.8	300.8	0.002	0.407	0.001	0.175	18698	38.8	0.011	3	735.2	804.9	56.9	0.002	NA	NA	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1734649	0	3016	699.7	1321.4	296.4	0.002	0.413	0.001	0.175	19227	38.7	0.011	3	735.2	804.9	56.8	0.002	NA	NA	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3047.40233.85418.7340.0100.0010.771
pg154_o3_native_lto.cx7a27_gcp_c2s3030.01540.77529.1980.0110.001nonzero1.092
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3032.66744.13723.1820.0130.001nonzero1.463
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3040.62532.18027.1870.0070.0010.858
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3037.23740.90621.8420.0130.001nonzero1.138

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3067.9711.7691.1933.60612.28413.1630.0150.408
pg154_o3_native_lto.cx7a27_gcp_c2s3064.00411.0900.9333.32913.2886.9720.3820.447
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3062.6474.9530.9893.47511.14016.7950.0010.497
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3064.1947.6030.9953.25310.43413.5190.0010.268
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3066.6936.3470.9253.23011.87210.934nonzero0.391

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
11426	0	7741	92.6	9964.0	143.6	0.677	8.297	0.872	12.869	20934	41.5	1.832	545	773.3	843.4	50.5	0.771	250	200	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
11103	0	10044	116.0	13326.3	177.8	0.905	10.697	1.200	16.402	25391	35.0	2.287	473	774.3	844.4	41.7	1.092	250	200	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
10124	0	8548	99.0	11530.6	158.6	0.844	10.014	1.139	16.042	21884	38.2	2.162	566	772.7	839.6	71.1	1.463	250	200	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
9564	0	7810	86.3	9878.0	129.8	0.817	9.239	1.033	13.899	20211	37.6	2.113	590	772.3	838.3	69.7	0.858	350	200	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
11241	0	9193	102.9	11887.7	158.7	0.818	9.372	1.058	14.457	23724	37.5	2.111	500	773.9	843.9	67.4	1.138	250	200	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.8250.1720.003nonzerononzerononzero0.113
pg154_o3_native_lto.cx7a27_gcp_c2s3099.7940.2040.002nonzerononzerononzero0.201
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.8550.1440.001nonzerononzerononzero0.099
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.8390.1590.001nonzerononzero0.025
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.8170.1820.001nonzero0.014

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3010.5567.23482.1880.0230.104
pg154_o3_native_lto.cx7a27_gcp_c2s306.09912.05681.8160.0300.249
pg16beta1_o3_native_lto.cx7a27_gcp_c2s301.64217.42780.9250.0050.082
pg16beta2_o3_native_lto.cx7a27_gcp_c2s303.72212.89283.3800.0050.100
pg16beta3_o3_native_lto.cx7a27_gcp_c2s309.5695.21485.2140.0030.067

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3041.21540.70717.1610.9150.0020.017
pg154_o3_native_lto.cx7a27_gcp_c2s3031.75939.79925.8612.5820.014
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3033.06137.99526.7142.2270.0030.024
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3030.32131.73134.4983.4500.013
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3039.62541.49017.3351.5470.0030.017

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
798	104467	1599	13.9	1719.1	24.2	0.015	0.136	2.156	31.133	400895	52.9	3.838	76	773.4	843.4	71.0	0.113	13074	6936	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
797	104603	1658	14.3	1911.9	22.3	0.016	0.140	2.398	28.644	401525	53.0	3.839	76	774.3	844.4	71.0	0.201	13185	12930	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
797	104678	1484	12.7	1552.0	21.2	0.014	0.124	1.947	27.187	401350	53.0	3.834	76	772.8	842.8	71.0	0.099	13153	11347	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
798	104990	1532	13.1	1300.2	19.7	0.015	0.128	1.630	25.244	402597	52.9	3.835	76	772.5	842.5	71.0	0.025	13217	10889	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
797	103679	1605	13.8	1508.2	19.3	0.015	0.136	1.892	24.819	397961	52.9	3.838	77	774.0	844.0	71.0	0.014	12994	11576	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.8750.1210.003nonzerononzero0.023
pg154_o3_native_lto.cx7a27_gcp_c2s3099.8530.1430.0040.001nonzero0.024
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.8360.1610.003nonzerononzero0.023
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.8230.1730.0040.001nonzero0.023
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.8410.1560.003nonzerononzero0.026

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s300.0488.83891.1140.050
pg154_o3_native_lto.cx7a27_gcp_c2s3011.73688.2640.056
pg16beta1_o3_native_lto.cx7a27_gcp_c2s304.28295.7180.057
pg16beta2_o3_native_lto.cx7a27_gcp_c2s306.98393.0150.0020.112
pg16beta3_o3_native_lto.cx7a27_gcp_c2s300.0785.58594.3360.057

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3064.8609.9782.84014.0738.2480.032
pg154_o3_native_lto.cx7a27_gcp_c2s3042.23110.1662.40823.40221.792nonzero0.067
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3064.46610.0374.25415.2525.9920.034
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3038.3899.6463.50126.06822.397nonzero0.064
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3065.65511.9312.29912.7217.3950.029

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3987	94699	5886	49.1	7390.2	85.9	0.062	0.530	1.854	22.066	367285	57.4	3.878	91	773.6	843.7	71.0	0.023	11833	11395	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
3988	96564	5892	49.0	7395.9	82.8	0.061	0.519	1.855	21.273	361761	59.0	3.746	92	774.5	844.5	71.0	0.024	12019	11315	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
3988	95672	5874	49.0	7368.9	94.7	0.061	0.524	1.848	24.330	371166	57.1	3.880	90	773.1	843.2	71.0	0.023	11859	11443	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
3987	97299	5856	48.7	7474.9	108.8	0.060	0.512	1.875	27.946	361957	59.1	3.720	91	773.1	843.1	71.0	0.023	12051	11459	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
3988	94216	5862	48.9	7332.3	88.9	0.062	0.531	1.839	22.834	365558	57.3	3.880	91	774.3	844.3	71.0	0.026	11619	11161	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3099.5490.4380.0100.003nonzero0.027
pg154_o3_native_lto.cx7a27_gcp_c2s3099.7030.2780.0150.005nonzero0.036
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3099.5250.4670.0070.002nonzero0.028
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3099.6590.3290.0080.003nonzerononzero0.079
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3099.6680.3220.0080.002nonzero0.027

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s309.46690.4640.0690.245
pg154_o3_native_lto.cx7a27_gcp_c2s3028.77771.1850.0380.0010.317
pg16beta1_o3_native_lto.cx7a27_gcp_c2s304.16095.8360.0040.173
pg16beta2_o3_native_lto.cx7a27_gcp_c2s309.82790.1650.0080.163
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3016.00883.9910.0010.105

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
pg153_o3_native_lto.cx7a27_gcp_c2s3050.48523.5020.1500.02925.8320.0020.086
pg154_o3_native_lto.cx7a27_gcp_c2s3015.71416.4430.1100.02061.1696.5440.149
pg16beta1_o3_native_lto.cx7a27_gcp_c2s3055.81927.2680.1610.02416.7140.0140.078
pg16beta2_o3_native_lto.cx7a27_gcp_c2s3048.06623.2930.2050.03228.3240.0810.084
pg16beta3_o3_native_lto.cx7a27_gcp_c2s3036.11226.1410.1640.04536.3571.1810.132

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7811	74007	11643	107.0	15912.2	185.1	0.157	1.481	2.037	24.267	274272	67.4	3.706	137	774.2	844.2	44.9	0.027	9254	8731	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
7209	95302	10771	98.7	14678.6	165.0	0.113	1.061	2.036	23.433	236040	82.8	2.477	130	774.8	844.9	44.9	0.036	12274	9594	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
7822	74696	11590	96.1	15640.7	194.3	0.155	1.318	2.000	25.435	291192	64.1	3.898	129	774.0	844.0	71.0	0.028	9382	8854	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
7617	77157	11242	93.1	15237.8	216.3	0.146	1.236	2.000	29.074	271760	66.8	3.522	130	774.4	844.4	71.0	0.079	9174	8694	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
7486	86676	11128	92.5	15034.8	174.5	0.128	1.093	2.008	23.867	260074	72.1	3.001	125	774.9	844.9	71.0	0.027	10644	8998	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i0

l.i0: load without secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1110802	0	540	4.3	1385.9	253.5	0.000	0.004	0.001	0.234	92534	84.6	0.083	11	382.7	452.8	9.9	0.662	142424	81810	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1101322	0	550	4.3	1364.3	250.3	0.000	0.004	0.001	0.233	91830	84.3	0.083	11	382.7	452.8	17.8	0.736	141645	71622	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1091405	0	532	4.2	1367.4	249.4	0.000	0.004	0.001	0.234	91678	83.9	0.084	12	382.7	452.8	13.3	0.604	140346	70224	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
1099505	0	548	4.3	1375.2	251.8	0.000	0.004	0.001	0.235	91448	84.4	0.083	12	382.7	452.8	5.4	0.561	141144	75418	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1109570	0	549	4.3	1373.8	254.3	0.000	0.004	0.001	0.235	91674	84.8	0.083	11	382.7	452.8	9.2	0.803	141946	92935	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

l.x: create secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
1624736	0	3555	768.1	1244.9	277.6	0.002	0.484	0.001	0.175	13241	37.1	0.008	3	735.2	804.7	56.7	0.002	NA	NA	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
1622759	0	3511	770.1	1243.1	280.0	0.002	0.486	0.001	0.177	13064	36.9	0.008	3	735.2	805.2	56.9	0.002	NA	NA	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
1739930	0	2974	687.3	1330.4	300.3	0.002	0.404	0.001	0.177	18486	39.0	0.011	3	735.2	804.8	57.2	0.002	NA	NA	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
1755200	0	3008	697.7	1344.8	300.8	0.002	0.407	0.001	0.175	18698	38.8	0.011	3	735.2	804.9	56.9	0.002	NA	NA	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
1734649	0	3016	699.7	1321.4	296.4	0.002	0.413	0.001	0.175	19227	38.7	0.011	3	735.2	804.9	56.8	0.002	NA	NA	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
11426	0	7741	92.6	9964.0	143.6	0.677	8.297	0.872	12.869	20934	41.5	1.832	545	773.3	843.4	50.5	0.771	250	200	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
11103	0	10044	116.0	13326.3	177.8	0.905	10.697	1.200	16.402	25391	35.0	2.287	473	774.3	844.4	41.7	1.092	250	200	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
10124	0	8548	99.0	11530.6	158.6	0.844	10.014	1.139	16.042	21884	38.2	2.162	566	772.7	839.6	71.1	1.463	250	200	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
9564	0	7810	86.3	9878.0	129.8	0.817	9.239	1.033	13.899	20211	37.6	2.113	590	772.3	838.3	69.7	0.858	350	200	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
11241	0	9193	102.9	11887.7	158.7	0.818	9.372	1.058	14.457	23724	37.5	2.111	500	773.9	843.9	67.4	1.138	250	200	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
798	104467	1599	13.9	1719.1	24.2	0.015	0.136	2.156	31.133	400895	52.9	3.838	76	773.4	843.4	71.0	0.113	13074	6936	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
797	104603	1658	14.3	1911.9	22.3	0.016	0.140	2.398	28.644	401525	53.0	3.839	76	774.3	844.4	71.0	0.201	13185	12930	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
797	104678	1484	12.7	1552.0	21.2	0.014	0.124	1.947	27.187	401350	53.0	3.834	76	772.8	842.8	71.0	0.099	13153	11347	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
798	104990	1532	13.1	1300.2	19.7	0.015	0.128	1.630	25.244	402597	52.9	3.835	76	772.5	842.5	71.0	0.025	13217	10889	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
797	103679	1605	13.8	1508.2	19.3	0.015	0.136	1.892	24.819	397961	52.9	3.838	77	774.0	844.0	71.0	0.014	12994	11576	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3987	94699	5886	49.1	7390.2	85.9	0.062	0.530	1.854	22.066	367285	57.4	3.878	91	773.6	843.7	71.0	0.023	11833	11395	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
3988	96564	5892	49.0	7395.9	82.8	0.061	0.519	1.855	21.273	361761	59.0	3.746	92	774.5	844.5	71.0	0.024	12019	11315	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
3988	95672	5874	49.0	7368.9	94.7	0.061	0.524	1.848	24.330	371166	57.1	3.880	90	773.1	843.2	71.0	0.023	11859	11443	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
3987	97299	5856	48.7	7474.9	108.8	0.060	0.512	1.875	27.946	361957	59.1	3.720	91	773.1	843.1	71.0	0.023	12051	11459	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
3988	94216	5862	48.9	7332.3	88.9	0.062	0.531	1.839	22.834	365558	57.3	3.880	91	774.3	844.3	71.0	0.026	11619	11161	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7811	74007	11643	107.0	15912.2	185.1	0.157	1.481	2.037	24.267	274272	67.4	3.706	137	774.2	844.2	44.9	0.027	9254	8731	4000m.pg153_o3_native_lto.cx7a27_gcp_c2s30
7209	95302	10771	98.7	14678.6	165.0	0.113	1.061	2.036	23.433	236040	82.8	2.477	130	774.8	844.9	44.9	0.036	12274	9594	4000m.pg154_o3_native_lto.cx7a27_gcp_c2s30
7822	74696	11590	96.1	15640.7	194.3	0.155	1.318	2.000	25.435	291192	64.1	3.898	129	774.0	844.0	71.0	0.028	9382	8854	4000m.pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
7617	77157	11242	93.1	15237.8	216.3	0.146	1.236	2.000	29.074	271760	66.8	3.522	130	774.4	844.4	71.0	0.079	9174	8694	4000m.pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
7486	86676	11128	92.5	15034.8	174.5	0.128	1.093	2.008	23.867	260074	72.1	3.001	125	774.9	844.9	71.0	0.027	10644	8998	4000m.pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.i0

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	97.643	2.315	0.029	0.005	0.008	0.001	0.000	0.000	0.000	0.662	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	97.550	2.408	0.028	0.005	0.009	0.001	0.000	0.000	0.000	0.736	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	97.414	2.545	0.027	0.005	0.008	0.001	0.000	0.000	0.000	0.604	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	97.593	2.366	0.029	0.005	0.007	0.001	0.000	0.000	0.000	0.561	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	97.519	2.444	0.027	0.005	0.006	nonzero	0.000	0.000	0.000	0.803	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

l.x

TODO - determine whether there is data for create index response time


l.i1

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	47.402	33.854	18.734	0.010	0.001	0.000	0.000	0.000	0.771	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	30.015	40.775	29.198	0.011	0.001	nonzero	0.000	0.000	1.092	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	32.667	44.137	23.182	0.013	0.001	nonzero	0.000	0.000	1.463	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	40.625	32.180	27.187	0.007	0.001	0.000	0.000	0.000	0.858	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	37.237	40.906	21.842	0.013	0.001	nonzero	0.000	0.000	1.138	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
67.971	1.769	1.193	3.606	12.284	13.163	0.015	0.000	0.000	0.000	0.408	pg153_o3_native_lto.cx7a27_gcp_c2s30
64.004	11.090	0.933	3.329	13.288	6.972	0.382	0.000	0.000	0.000	0.447	pg154_o3_native_lto.cx7a27_gcp_c2s30
62.647	4.953	0.989	3.475	11.140	16.795	0.001	0.000	0.000	0.000	0.497	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
64.194	7.603	0.995	3.253	10.434	13.519	0.001	0.000	0.000	0.000	0.268	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
66.693	6.347	0.925	3.230	11.872	10.934	nonzero	0.000	0.000	0.000	0.391	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q100.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.825	0.172	0.003	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.113	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.794	0.204	0.002	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.201	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.855	0.144	0.001	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.099	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.839	0.159	0.001	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.025	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.817	0.182	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.000	0.014	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	10.556	7.234	82.188	0.023	0.000	0.000	0.000	0.000	0.104	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	6.099	12.056	81.816	0.030	0.000	0.000	0.000	0.000	0.249	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	1.642	17.427	80.925	0.005	0.000	0.000	0.000	0.000	0.082	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	3.722	12.892	83.380	0.005	0.000	0.000	0.000	0.000	0.100	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	9.569	5.214	85.214	0.003	0.000	0.000	0.000	0.000	0.067	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
41.215	40.707	17.161	0.915	0.002	0.000	0.000	0.000	0.000	0.000	0.017	pg153_o3_native_lto.cx7a27_gcp_c2s30
31.759	39.799	25.861	2.582	0.000	0.000	0.000	0.000	0.000	0.000	0.014	pg154_o3_native_lto.cx7a27_gcp_c2s30
33.061	37.995	26.714	2.227	0.003	0.000	0.000	0.000	0.000	0.000	0.024	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
30.321	31.731	34.498	3.450	0.000	0.000	0.000	0.000	0.000	0.000	0.013	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
39.625	41.490	17.335	1.547	0.003	0.000	0.000	0.000	0.000	0.000	0.017	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q500.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.875	0.121	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.023	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.853	0.143	0.004	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.024	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.836	0.161	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.023	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.823	0.173	0.004	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.023	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.841	0.156	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.000	0.026	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	0.048	8.838	91.114	0.000	0.000	0.000	0.000	0.000	0.050	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	11.736	88.264	0.000	0.000	0.000	0.000	0.000	0.056	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	4.282	95.718	0.000	0.000	0.000	0.000	0.000	0.057	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	6.983	93.015	0.002	0.000	0.000	0.000	0.000	0.112	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.078	5.585	94.336	0.000	0.000	0.000	0.000	0.000	0.057	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
64.860	9.978	2.840	14.073	8.248	0.000	0.000	0.000	0.000	0.000	0.032	pg153_o3_native_lto.cx7a27_gcp_c2s30
42.231	10.166	2.408	23.402	21.792	nonzero	0.000	0.000	0.000	0.000	0.067	pg154_o3_native_lto.cx7a27_gcp_c2s30
64.466	10.037	4.254	15.252	5.992	0.000	0.000	0.000	0.000	0.000	0.034	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
38.389	9.646	3.501	26.068	22.397	nonzero	0.000	0.000	0.000	0.000	0.064	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
65.655	11.931	2.299	12.721	7.395	0.000	0.000	0.000	0.000	0.000	0.029	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

q1000.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.549	0.438	0.010	0.003	nonzero	0.000	0.000	0.000	0.000	0.000	0.027	pg153_o3_native_lto.cx7a27_gcp_c2s30
99.703	0.278	0.015	0.005	nonzero	0.000	0.000	0.000	0.000	0.000	0.036	pg154_o3_native_lto.cx7a27_gcp_c2s30
99.525	0.467	0.007	0.002	nonzero	0.000	0.000	0.000	0.000	0.000	0.028	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
99.659	0.329	0.008	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.079	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
99.668	0.322	0.008	0.002	nonzero	0.000	0.000	0.000	0.000	0.000	0.027	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	0.000	9.466	90.464	0.069	0.000	0.000	0.000	0.000	0.245	pg153_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	28.777	71.185	0.038	0.001	0.000	0.000	0.000	0.317	pg154_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	4.160	95.836	0.004	0.000	0.000	0.000	0.000	0.173	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	9.827	90.165	0.008	0.000	0.000	0.000	0.000	0.163	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
0.000	0.000	0.000	16.008	83.991	0.001	0.000	0.000	0.000	0.000	0.105	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
50.485	23.502	0.150	0.029	25.832	0.002	0.000	0.000	0.000	0.000	0.086	pg153_o3_native_lto.cx7a27_gcp_c2s30
15.714	16.443	0.110	0.020	61.169	6.544	0.000	0.000	0.000	0.000	0.149	pg154_o3_native_lto.cx7a27_gcp_c2s30
55.819	27.268	0.161	0.024	16.714	0.014	0.000	0.000	0.000	0.000	0.078	pg16beta1_o3_native_lto.cx7a27_gcp_c2s30
48.066	23.293	0.205	0.032	28.324	0.081	0.000	0.000	0.000	0.000	0.084	pg16beta2_o3_native_lto.cx7a27_gcp_c2s30
36.112	26.141	0.164	0.045	36.357	1.181	0.000	0.000	0.000	0.000	0.132	pg16beta3_o3_native_lto.cx7a27_gcp_c2s30