Introduction

This is a report for the insert benchmark with 160M docs and 8 client(s). It is generated by scripts (bash, awk, sed) and Tufte might not be impressed. An overview of the insert benchmark is here and a short update is here. Below, by DBMS, I mean DBMS+version.config. An example is my8020.c10b40 where my means MySQL, 8020 is version 8.0.20 and c10b40 is the name for the configuration file.

The test server is a c2-standard-30 from GCP with 15 cores, hyperthreads disabled, 120G RAM, XFS + SW RAID 0 on 4 NVMe devices (1.5TB). The benchmark was run with 8 client and there were 1 or 3 connections per client (1 for queries or inserts without rate limits, 1+1 for rate limited inserts+deletes). There are 8 tables, client per table. It loads 20M rows without secondary indexes, creates secondary indexes, then inserts 200M rows with a delete per insert to avoid growing the table. It then does 3 read+write tests for 7200s each that do queries as fast as possible with 100, 500 and then 1000 inserts/second/client concurrent with the queries and 1000 deletes/second to avoid growing the table. The database is cached by InnoDB.

The tested DBMS are:


Contents


Summary

The numbers are inserts/s for l.i0 and l.i1, indexed docs (or rows) /s for l.x and queries/s for q*.2. The values are the average rate over the entire test for inserts (IPS) and queries (QPS). The range of values for IPS and QPS is split into 3 parts: bottom 25%, middle 50%, top 25%. Values in the bottom 25% have a red background, values in the top 25% have a green background and values in the middle have no color. A gray background is used for values that can be ignored because the DBMS did not sustain the target insert rate. Red backgrounds are not used when the minimum value is within 80% of the max value.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
my5651_rel.cy10a_gcp_c2s30 41237159738875404444214519255777
my5740_rel_lto.cy10a_gcp_c2s30 7766991168613126753370363869239205
my8033_rel_lto.cy10a_gcp_c2s30 666667884530120782380745064050829
ps8033_rel_lto.cy10aps1_gcp_c2s30 666667879670119922361924797949519
ps8033_rel_lto.cy10aps2_gcp_c2s30 663900847090119859360964787049480

This table has relative throughput, throughput for the DBMS relative to the DBMS in the first line, using the absolute throughput from the previous table. Values less than 0.95 have a yellow background. Values greater than 1.05 have a blue background.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
my5651_rel.cy10a_gcp_c2s30 1.001.001.001.001.001.00
my5740_rel_lto.cy10a_gcp_c2s30 1.881.961.680.830.860.70
my8033_rel_lto.cy10a_gcp_c2s30 1.621.481.600.861.120.91
ps8033_rel_lto.cy10aps1_gcp_c2s30 1.621.471.590.811.060.89
ps8033_rel_lto.cy10aps2_gcp_c2s30 1.611.421.590.811.060.89

This lists the average rate of inserts/s for the tests that do inserts concurrent with queries. For such tests the query rate is listed in the table above. The read+write tests are setup so that the insert rate should match the target rate every second. Cells that are not at least 95% of the target have a red background to indicate a failure to satisfy the target.

dbmsq100.1q500.1q1000.1
my5651_rel.cy10a_gcp_c2s3075238007848
my5740_rel_lto.cy10a_gcp_c2s3078239297850
my8033_rel_lto.cy10a_gcp_c2s3078939917984
ps8033_rel_lto.cy10aps1_gcp_c2s3078839897984
ps8033_rel_lto.cy10aps2_gcp_c2s3079039907984
target80040008000

l.i0

l.i0: load without secondary indexes. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s302.19997.5060.1140.1680.0140.195
my5740_rel_lto.cy10a_gcp_c2s3094.4065.5290.0310.0190.0150.244
my8033_rel_lto.cy10a_gcp_c2s308.36491.5560.0400.0260.0140.232
ps8033_rel_lto.cy10aps1_gcp_c2s3010.76289.1560.0420.0260.0140.236
ps8033_rel_lto.cy10aps2_gcp_c2s308.20891.7100.0410.0270.0140.231

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
412371	0	3251	12.7	550.4	73.6	0.008	0.032	0.001	0.183	84759	43.5	0.206	16	10.6	139.1	16.1	0.195	53043	42754	160m.my5651_rel.cy10a_gcp_c2s30
776699	0	0	0.0	1450.7	150.2	0.000	0.000	0.002	0.198	76832	70.3	0.099	14	10.6	139.1	17.2	0.244	107319	85407	160m.my5740_rel_lto.cy10a_gcp_c2s30
666667	0	0	0.0	1112.1	123.1	0.000	0.000	0.002	0.189	206689	72.8	0.310	16	10.6	139.1	17.2	0.232	90701	71720	160m.my8033_rel_lto.cy10a_gcp_c2s30
666667	0	0	0.0	895.9	120.3	0.000	0.000	0.001	0.185	215257	72.8	0.323	16	10.6	139.1	17.2	0.236	90634	73945	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
663900	0	0	0.0	891.4	120.8	0.000	0.000	0.001	0.186	205401	73.2	0.309	17	10.6	139.1	17.1	0.231	90301	71282	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.x

l.x: create secondary indexes.

Average throughput:

Image

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
597388	0	318	1.2	7947.4	630.6	0.001	0.002	0.013	1.081	18514	39.1	0.031	10	21.7	150.2	25.9	0.001	NA	NA	160m.my5651_rel.cy10a_gcp_c2s30
1168613	0	0	0.0	13965.2	1060.7	0.000	0.000	0.012	0.929	47770	37.7	0.041	5	23.6	152.1	25.2	0.031	NA	NA	160m.my5740_rel_lto.cy10a_gcp_c2s30
884530	0	4451	282.9	18070.7	868.1	0.005	0.328	0.020	1.005	111178	81.1	0.126	14	23.5	152.1	26.7	0.011	NA	NA	160m.my8033_rel_lto.cy10a_gcp_c2s30
879670	0	4453	282.9	17863.6	862.7	0.005	0.329	0.020	1.004	105895	82.8	0.120	14	23.5	152.1	25.8	0.004	NA	NA	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
847090	0	4330	275.3	17306.8	837.4	0.005	0.333	0.020	1.012	109796	82.0	0.130	15	23.5	152.1	26.4	0.004	NA	NA	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3019.29280.3100.3810.016nonzero0.262
my5740_rel_lto.cy10a_gcp_c2s3084.91815.0160.0510.016nonzero0.390
my8033_rel_lto.cy10a_gcp_c2s3082.36817.5490.0640.0180.001nonzero1.015
ps8033_rel_lto.cy10aps1_gcp_c2s3081.81818.0860.0770.0160.0030.742
ps8033_rel_lto.cy10aps2_gcp_c2s3081.63518.2760.0710.018nonzero0.866

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s30nonzero40.15559.4910.3400.014nonzero0.258
my5740_rel_lto.cy10a_gcp_c2s30nonzero86.12613.7990.0580.017nonzero0.405
my8033_rel_lto.cy10a_gcp_c2s3084.28915.6300.0660.014nonzerononzero1.019
ps8033_rel_lto.cy10aps1_gcp_c2s3083.52316.3760.0860.0120.0030.679
ps8033_rel_lto.cy10aps2_gcp_c2s3083.39716.5150.0730.015nonzero0.865

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
75404	0	3519	28.5	8030.7	275.3	0.047	0.387	0.107	3.738	167646	61.7	2.223	123	179.5	379.3	85.7	0.262	9490	7941	160m.my5651_rel.cy10a_gcp_c2s30
126753	0	2212	34.6	11359.1	397.8	0.017	0.279	0.090	3.213	67655	85.7	0.534	101	200.2	410.4	87.0	0.390	16089	11837	160m.my5740_rel_lto.cy10a_gcp_c2s30
120782	0	873	13.6	9359.0	324.0	0.007	0.116	0.077	2.747	169939	86.0	1.407	107	148.2	334.7	86.9	1.015	15283	11587	160m.my8033_rel_lto.cy10a_gcp_c2s30
119922	0	925	14.5	8504.1	299.7	0.008	0.123	0.071	2.559	166548	86.0	1.389	108	149.9	332.2	86.9	0.742	15183	11188	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
119859	0	914	14.3	8493.4	299.6	0.008	0.122	0.071	2.559	165056	86.1	1.377	108	149.9	333.3	86.9	0.866	15141	11587	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3097.9231.9120.1630.003nonzerononzero0.256
my5740_rel_lto.cy10a_gcp_c2s3090.2718.9220.8040.003nonzerononzero0.224
my8033_rel_lto.cy10a_gcp_c2s3091.6468.3000.054nonzerononzerononzero0.247
ps8033_rel_lto.cy10aps1_gcp_c2s3086.50713.3800.113nonzerononzerononzero0.233
ps8033_rel_lto.cy10aps2_gcp_c2s3086.15013.7560.094nonzerononzerononzero0.227

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3095.6394.3240.0190.0180.205
my5740_rel_lto.cy10a_gcp_c2s3081.82718.1610.0120.028
my8033_rel_lto.cy10a_gcp_c2s3095.5724.4160.0100.0010.0010.457
ps8033_rel_lto.cy10aps1_gcp_c2s3096.8293.1590.0120.043
ps8033_rel_lto.cy10aps2_gcp_c2s3095.2344.7490.0160.050

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3096.7353.2420.0060.0160.189
my5740_rel_lto.cy10a_gcp_c2s3099.4510.5490.0010.017
my8033_rel_lto.cy10a_gcp_c2s3096.3653.6270.0070.0010.0010.457
ps8033_rel_lto.cy10aps1_gcp_c2s3097.3562.6350.0090.043
ps8033_rel_lto.cy10aps2_gcp_c2s3096.0533.9360.0110.050

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
752	44421	9859	153.7	14838.0	424.6	0.222	3.543	19.731	578.226	212156	58.3	4.776	197	179.5	379.3	85.8	0.256	5961	2829	160m.my5651_rel.cy10a_gcp_c2s30
782	37036	13178	205.9	14675.6	434.9	0.356	5.693	18.760	569.213	196533	59.0	5.306	239	200.2	410.4	87.0	0.224	4827	1295	160m.my5740_rel_lto.cy10a_gcp_c2s30
789	38074	2445	38.2	7082.7	198.2	0.064	1.028	8.981	257.334	154933	70.4	4.069	277	148.2	301.9	87.0	0.247	4923	3868	160m.my8033_rel_lto.cy10a_gcp_c2s30
788	36192	2986	46.7	7248.6	208.0	0.083	1.320	9.196	270.246	154294	69.3	4.263	287	149.9	292.6	87.0	0.233	4683	2382	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
790	36096	2963	46.3	7189.8	206.7	0.082	1.313	9.106	268.059	151312	69.9	4.192	290	149.9	295.2	87.0	0.227	4667	2574	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3098.1001.7970.0990.003nonzerononzerononzero0.257
my5740_rel_lto.cy10a_gcp_c2s3091.5838.1290.2850.002nonzerononzero0.245
my8033_rel_lto.cy10a_gcp_c2s3099.2350.7570.0080.001nonzero0.022
ps8033_rel_lto.cy10aps1_gcp_c2s3098.6851.3060.0080.001nonzero0.056
ps8033_rel_lto.cy10aps2_gcp_c2s3098.6821.3070.0100.002nonzero0.026

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3096.5983.3590.0420.040
my5740_rel_lto.cy10a_gcp_c2s3099.2880.6860.0260.035
my8033_rel_lto.cy10a_gcp_c2s3097.8972.0720.0300.0010.177
ps8033_rel_lto.cy10aps1_gcp_c2s3096.4643.4890.046nonzero0.076
ps8033_rel_lto.cy10aps2_gcp_c2s3093.6616.1810.1580.036

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3097.9312.0640.0050.038
my5740_rel_lto.cy10a_gcp_c2s3099.6320.3630.0050.026
my8033_rel_lto.cy10a_gcp_c2s3098.5691.4100.0200.0010.177
ps8033_rel_lto.cy10aps1_gcp_c2s3097.6132.3570.030nonzero0.082
ps8033_rel_lto.cy10aps2_gcp_c2s3095.0904.7920.1180.034

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3800	45192	7653	118.0	16171.8	483.7	0.169	2.674	4.255	130.335	206121	63.0	4.561	209	179.5	379.3	85.8	0.257	5900	5082	160m.my5651_rel.cy10a_gcp_c2s30
3929	38692	10756	168.1	13956.3	417.2	0.278	4.448	3.552	108.722	189641	63.9	4.901	248	200.2	410.4	87.0	0.245	4938	4093	160m.my5740_rel_lto.cy10a_gcp_c2s30
3991	50640	128	2.0	994.2	30.6	0.003	0.040	0.249	7.842	196716	58.6	3.885	174	148.2	277.3	87.0	0.022	6511	5101	160m.my8033_rel_lto.cy10a_gcp_c2s30
3989	47979	221	3.5	1033.9	32.8	0.005	0.074	0.259	8.407	184380	60.7	3.843	190	149.9	279.0	87.0	0.056	6313	4907	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
3990	47870	198	3.1	994.0	31.3	0.004	0.066	0.249	8.033	184501	60.0	3.854	188	149.9	279.0	87.0	0.026	6265	4891	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s3099.1500.7520.0840.0130.001nonzero0.191
my5740_rel_lto.cy10a_gcp_c2s3093.0476.8100.1410.0020.001nonzerononzero0.311
my8033_rel_lto.cy10a_gcp_c2s3099.3780.5920.0260.004nonzero0.031
ps8033_rel_lto.cy10aps1_gcp_c2s3099.1840.7830.0290.004nonzero0.044
ps8033_rel_lto.cy10aps2_gcp_c2s3099.1880.7780.0300.004nonzero0.030

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s30nonzero91.7008.2400.0600.048
my5740_rel_lto.cy10a_gcp_c2s3099.2510.7140.0350.052
my8033_rel_lto.cy10a_gcp_c2s3087.42112.0930.4850.038
ps8033_rel_lto.cy10aps1_gcp_c2s3085.84113.5250.6340.047
ps8033_rel_lto.cy10aps2_gcp_c2s3085.13014.2990.5720.046

Delete response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy10a_gcp_c2s300.08394.6485.2510.0180.032
my5740_rel_lto.cy10a_gcp_c2s3099.5870.4080.0060.035
my8033_rel_lto.cy10a_gcp_c2s3089.9349.7020.3640.048
ps8033_rel_lto.cy10aps1_gcp_c2s3088.36511.1430.4920.050
ps8033_rel_lto.cy10aps2_gcp_c2s3087.84711.7260.4270.039

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7848	55777	1407	18.9	4665.1	145.4	0.025	0.348	0.594	18.965	199465	68.5	3.576	184	179.5	379.3	85.8	0.191	6952	5850	160m.my5651_rel.cy10a_gcp_c2s30
7850	39205	8521	133.1	13328.0	398.4	0.217	3.478	1.698	51.976	179415	67.8	4.576	259	200.2	410.4	86.9	0.311	5034	4123	160m.my5740_rel_lto.cy10a_gcp_c2s30
7984	50829	40	0.6	780.2	25.0	0.001	0.013	0.098	3.200	194464	59.0	3.826	174	148.2	277.3	87.0	0.031	6363	6258	160m.my8033_rel_lto.cy10a_gcp_c2s30
7984	49519	40	0.6	487.7	16.9	0.001	0.013	0.061	2.164	188827	59.0	3.813	179	149.9	279.0	87.0	0.044	6201	6105	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
7984	49480	40	0.6	486.5	16.9	0.001	0.013	0.061	2.163	188710	58.8	3.814	178	149.9	279.0	87.0	0.030	6191	6087	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.i0

l.i0: load without secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
412371	0	3251	12.7	550.4	73.6	0.008	0.032	0.001	0.183	84759	43.5	0.206	16	10.6	139.1	16.1	0.195	53043	42754	160m.my5651_rel.cy10a_gcp_c2s30
776699	0	0	0.0	1450.7	150.2	0.000	0.000	0.002	0.198	76832	70.3	0.099	14	10.6	139.1	17.2	0.244	107319	85407	160m.my5740_rel_lto.cy10a_gcp_c2s30
666667	0	0	0.0	1112.1	123.1	0.000	0.000	0.002	0.189	206689	72.8	0.310	16	10.6	139.1	17.2	0.232	90701	71720	160m.my8033_rel_lto.cy10a_gcp_c2s30
666667	0	0	0.0	895.9	120.3	0.000	0.000	0.001	0.185	215257	72.8	0.323	16	10.6	139.1	17.2	0.236	90634	73945	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
663900	0	0	0.0	891.4	120.8	0.000	0.000	0.001	0.186	205401	73.2	0.309	17	10.6	139.1	17.1	0.231	90301	71282	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.x

l.x: create secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
597388	0	318	1.2	7947.4	630.6	0.001	0.002	0.013	1.081	18514	39.1	0.031	10	21.7	150.2	25.9	0.001	NA	NA	160m.my5651_rel.cy10a_gcp_c2s30
1168613	0	0	0.0	13965.2	1060.7	0.000	0.000	0.012	0.929	47770	37.7	0.041	5	23.6	152.1	25.2	0.031	NA	NA	160m.my5740_rel_lto.cy10a_gcp_c2s30
884530	0	4451	282.9	18070.7	868.1	0.005	0.328	0.020	1.005	111178	81.1	0.126	14	23.5	152.1	26.7	0.011	NA	NA	160m.my8033_rel_lto.cy10a_gcp_c2s30
879670	0	4453	282.9	17863.6	862.7	0.005	0.329	0.020	1.004	105895	82.8	0.120	14	23.5	152.1	25.8	0.004	NA	NA	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
847090	0	4330	275.3	17306.8	837.4	0.005	0.333	0.020	1.012	109796	82.0	0.130	15	23.5	152.1	26.4	0.004	NA	NA	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.i1

l.i1: continue load after secondary indexes created

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
75404	0	3519	28.5	8030.7	275.3	0.047	0.387	0.107	3.738	167646	61.7	2.223	123	179.5	379.3	85.7	0.262	9490	7941	160m.my5651_rel.cy10a_gcp_c2s30
126753	0	2212	34.6	11359.1	397.8	0.017	0.279	0.090	3.213	67655	85.7	0.534	101	200.2	410.4	87.0	0.390	16089	11837	160m.my5740_rel_lto.cy10a_gcp_c2s30
120782	0	873	13.6	9359.0	324.0	0.007	0.116	0.077	2.747	169939	86.0	1.407	107	148.2	334.7	86.9	1.015	15283	11587	160m.my8033_rel_lto.cy10a_gcp_c2s30
119922	0	925	14.5	8504.1	299.7	0.008	0.123	0.071	2.559	166548	86.0	1.389	108	149.9	332.2	86.9	0.742	15183	11188	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
119859	0	914	14.3	8493.4	299.6	0.008	0.122	0.071	2.559	165056	86.1	1.377	108	149.9	333.3	86.9	0.866	15141	11587	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q100.1

q100.1: range queries with 100 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
752	44421	9859	153.7	14838.0	424.6	0.222	3.543	19.731	578.226	212156	58.3	4.776	197	179.5	379.3	85.8	0.256	5961	2829	160m.my5651_rel.cy10a_gcp_c2s30
782	37036	13178	205.9	14675.6	434.9	0.356	5.693	18.760	569.213	196533	59.0	5.306	239	200.2	410.4	87.0	0.224	4827	1295	160m.my5740_rel_lto.cy10a_gcp_c2s30
789	38074	2445	38.2	7082.7	198.2	0.064	1.028	8.981	257.334	154933	70.4	4.069	277	148.2	301.9	87.0	0.247	4923	3868	160m.my8033_rel_lto.cy10a_gcp_c2s30
788	36192	2986	46.7	7248.6	208.0	0.083	1.320	9.196	270.246	154294	69.3	4.263	287	149.9	292.6	87.0	0.233	4683	2382	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
790	36096	2963	46.3	7189.8	206.7	0.082	1.313	9.106	268.059	151312	69.9	4.192	290	149.9	295.2	87.0	0.227	4667	2574	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q500.1

q500.1: range queries with 500 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
3800	45192	7653	118.0	16171.8	483.7	0.169	2.674	4.255	130.335	206121	63.0	4.561	209	179.5	379.3	85.8	0.257	5900	5082	160m.my5651_rel.cy10a_gcp_c2s30
3929	38692	10756	168.1	13956.3	417.2	0.278	4.448	3.552	108.722	189641	63.9	4.901	248	200.2	410.4	87.0	0.245	4938	4093	160m.my5740_rel_lto.cy10a_gcp_c2s30
3991	50640	128	2.0	994.2	30.6	0.003	0.040	0.249	7.842	196716	58.6	3.885	174	148.2	277.3	87.0	0.022	6511	5101	160m.my8033_rel_lto.cy10a_gcp_c2s30
3989	47979	221	3.5	1033.9	32.8	0.005	0.074	0.259	8.407	184380	60.7	3.843	190	149.9	279.0	87.0	0.056	6313	4907	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
3990	47870	198	3.1	994.0	31.3	0.004	0.066	0.249	8.033	184501	60.0	3.854	188	149.9	279.0	87.0	0.026	6265	4891	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

q1000.1

q1000.1: range queries with 1000 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
7848	55777	1407	18.9	4665.1	145.4	0.025	0.348	0.594	18.965	199465	68.5	3.576	184	179.5	379.3	85.8	0.191	6952	5850	160m.my5651_rel.cy10a_gcp_c2s30
7850	39205	8521	133.1	13328.0	398.4	0.217	3.478	1.698	51.976	179415	67.8	4.576	259	200.2	410.4	86.9	0.311	5034	4123	160m.my5740_rel_lto.cy10a_gcp_c2s30
7984	50829	40	0.6	780.2	25.0	0.001	0.013	0.098	3.200	194464	59.0	3.826	174	148.2	277.3	87.0	0.031	6363	6258	160m.my8033_rel_lto.cy10a_gcp_c2s30
7984	49519	40	0.6	487.7	16.9	0.001	0.013	0.061	2.164	188827	59.0	3.813	179	149.9	279.0	87.0	0.044	6201	6105	160m.ps8033_rel_lto.cy10aps1_gcp_c2s30
7984	49480	40	0.6	486.5	16.9	0.001	0.013	0.061	2.163	188710	58.8	3.814	178	149.9	279.0	87.0	0.030	6191	6087	160m.ps8033_rel_lto.cy10aps2_gcp_c2s30

l.i0

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	2.199	97.506	0.114	0.168	0.014	0.000	0.000	0.000	0.000	0.195	my5651_rel.cy10a_gcp_c2s30
0.000	94.406	5.529	0.031	0.019	0.015	0.000	0.000	0.000	0.000	0.244	my5740_rel_lto.cy10a_gcp_c2s30
0.000	8.364	91.556	0.040	0.026	0.014	0.000	0.000	0.000	0.000	0.232	my8033_rel_lto.cy10a_gcp_c2s30
0.000	10.762	89.156	0.042	0.026	0.014	0.000	0.000	0.000	0.000	0.236	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	8.208	91.710	0.041	0.027	0.014	0.000	0.000	0.000	0.000	0.231	ps8033_rel_lto.cy10aps2_gcp_c2s30

l.x

TODO - determine whether there is data for create index response time


l.i1

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	19.292	80.310	0.381	0.016	nonzero	0.000	0.000	0.000	0.262	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	84.918	15.016	0.051	0.016	nonzero	0.000	0.000	0.000	0.390	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	82.368	17.549	0.064	0.018	0.001	nonzero	0.000	0.000	1.015	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	81.818	18.086	0.077	0.016	0.003	0.000	0.000	0.000	0.742	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	81.635	18.276	0.071	0.018	nonzero	0.000	0.000	0.000	0.866	ps8033_rel_lto.cy10aps2_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	nonzero	40.155	59.491	0.340	0.014	nonzero	0.000	0.000	0.000	0.258	my5651_rel.cy10a_gcp_c2s30
0.000	nonzero	86.126	13.799	0.058	0.017	nonzero	0.000	0.000	0.000	0.405	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	84.289	15.630	0.066	0.014	nonzero	nonzero	0.000	0.000	1.019	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	83.523	16.376	0.086	0.012	0.003	0.000	0.000	0.000	0.679	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	83.397	16.515	0.073	0.015	nonzero	0.000	0.000	0.000	0.865	ps8033_rel_lto.cy10aps2_gcp_c2s30

q100.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
97.923	1.912	0.163	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.256	my5651_rel.cy10a_gcp_c2s30
90.271	8.922	0.804	0.003	nonzero	nonzero	0.000	0.000	0.000	0.000	0.224	my5740_rel_lto.cy10a_gcp_c2s30
91.646	8.300	0.054	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.247	my8033_rel_lto.cy10a_gcp_c2s30
86.507	13.380	0.113	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.233	ps8033_rel_lto.cy10aps1_gcp_c2s30
86.150	13.756	0.094	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.000	0.227	ps8033_rel_lto.cy10aps2_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	95.639	4.324	0.019	0.018	0.000	0.000	0.000	0.000	0.205	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	81.827	18.161	0.012	0.000	0.000	0.000	0.000	0.000	0.028	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	95.572	4.416	0.010	0.001	0.001	0.000	0.000	0.000	0.457	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	96.829	3.159	0.012	0.000	0.000	0.000	0.000	0.000	0.043	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	95.234	4.749	0.016	0.000	0.000	0.000	0.000	0.000	0.050	ps8033_rel_lto.cy10aps2_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	96.735	3.242	0.006	0.016	0.000	0.000	0.000	0.000	0.189	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	99.451	0.549	0.001	0.000	0.000	0.000	0.000	0.000	0.017	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	96.365	3.627	0.007	0.001	0.001	0.000	0.000	0.000	0.457	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	97.356	2.635	0.009	0.000	0.000	0.000	0.000	0.000	0.043	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	96.053	3.936	0.011	0.000	0.000	0.000	0.000	0.000	0.050	ps8033_rel_lto.cy10aps2_gcp_c2s30

q500.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
98.100	1.797	0.099	0.003	nonzero	nonzero	nonzero	0.000	0.000	0.000	0.257	my5651_rel.cy10a_gcp_c2s30
91.583	8.129	0.285	0.002	nonzero	nonzero	0.000	0.000	0.000	0.000	0.245	my5740_rel_lto.cy10a_gcp_c2s30
99.235	0.757	0.008	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.022	my8033_rel_lto.cy10a_gcp_c2s30
98.685	1.306	0.008	0.001	nonzero	0.000	0.000	0.000	0.000	0.000	0.056	ps8033_rel_lto.cy10aps1_gcp_c2s30
98.682	1.307	0.010	0.002	nonzero	0.000	0.000	0.000	0.000	0.000	0.026	ps8033_rel_lto.cy10aps2_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	96.598	3.359	0.042	0.000	0.000	0.000	0.000	0.000	0.040	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	99.288	0.686	0.026	0.000	0.000	0.000	0.000	0.000	0.035	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	97.897	2.072	0.030	0.001	0.000	0.000	0.000	0.000	0.177	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	96.464	3.489	0.046	nonzero	0.000	0.000	0.000	0.000	0.076	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	93.661	6.181	0.158	0.000	0.000	0.000	0.000	0.000	0.036	ps8033_rel_lto.cy10aps2_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	97.931	2.064	0.005	0.000	0.000	0.000	0.000	0.000	0.038	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	99.632	0.363	0.005	0.000	0.000	0.000	0.000	0.000	0.026	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	98.569	1.410	0.020	0.001	0.000	0.000	0.000	0.000	0.177	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	97.613	2.357	0.030	nonzero	0.000	0.000	0.000	0.000	0.082	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	95.090	4.792	0.118	0.000	0.000	0.000	0.000	0.000	0.034	ps8033_rel_lto.cy10aps2_gcp_c2s30

q1000.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
99.150	0.752	0.084	0.013	0.001	nonzero	0.000	0.000	0.000	0.000	0.191	my5651_rel.cy10a_gcp_c2s30
93.047	6.810	0.141	0.002	0.001	nonzero	nonzero	0.000	0.000	0.000	0.311	my5740_rel_lto.cy10a_gcp_c2s30
99.378	0.592	0.026	0.004	nonzero	0.000	0.000	0.000	0.000	0.000	0.031	my8033_rel_lto.cy10a_gcp_c2s30
99.184	0.783	0.029	0.004	nonzero	0.000	0.000	0.000	0.000	0.000	0.044	ps8033_rel_lto.cy10aps1_gcp_c2s30
99.188	0.778	0.030	0.004	nonzero	0.000	0.000	0.000	0.000	0.000	0.030	ps8033_rel_lto.cy10aps2_gcp_c2s30

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	nonzero	91.700	8.240	0.060	0.000	0.000	0.000	0.000	0.000	0.048	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	99.251	0.714	0.035	0.000	0.000	0.000	0.000	0.000	0.052	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	87.421	12.093	0.485	0.000	0.000	0.000	0.000	0.000	0.038	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	85.841	13.525	0.634	0.000	0.000	0.000	0.000	0.000	0.047	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	85.130	14.299	0.572	0.000	0.000	0.000	0.000	0.000	0.046	ps8033_rel_lto.cy10aps2_gcp_c2s30

Delete response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.083	94.648	5.251	0.018	0.000	0.000	0.000	0.000	0.000	0.032	my5651_rel.cy10a_gcp_c2s30
0.000	0.000	99.587	0.408	0.006	0.000	0.000	0.000	0.000	0.000	0.035	my5740_rel_lto.cy10a_gcp_c2s30
0.000	0.000	89.934	9.702	0.364	0.000	0.000	0.000	0.000	0.000	0.048	my8033_rel_lto.cy10a_gcp_c2s30
0.000	0.000	88.365	11.143	0.492	0.000	0.000	0.000	0.000	0.000	0.050	ps8033_rel_lto.cy10aps1_gcp_c2s30
0.000	0.000	87.847	11.726	0.427	0.000	0.000	0.000	0.000	0.000	0.039	ps8033_rel_lto.cy10aps2_gcp_c2s30