Introduction

This is a report for the insert benchmark with 20M docs and 1 client(s). It is generated by scripts (bash, awk, sed) and Tufte might not be impressed. An overview of the insert benchmark is here and a short update is here. Below, by DBMS, I mean DBMS+version.config. An example is my8020.c10b40 where my means MySQL, 8020 is version 8.0.20 and c10b40 is the name for the configuration file.

The test server has 8 AMD cores, 16G RAM and an NVMe SSD. It is described here as the Beelink. The benchmark was run with 1 client and there were 1 or 2 connections per client (1 for queries, 1 for inserts). The benchmark loads 20M rows without secondary indexes, creates secondary indexes, loads another 20M rows then does 3 read+write tests for one hour each that do queries as fast as possible with 100, 500 and then 1000 writes/second/client concurrent with the queries. There is 1 table. The database is cached by the OS but not by InnoDB. Clients and the DBMS share one server. The per-database configs are in the per-database subdirectories here.

The tested DBMS are:


Contents


Summary

The numbers are inserts/s for l.i0 and l.i1, indexed docs (or rows) /s for l.x and queries/s for q*.2. The values are the average rate over the entire test for inserts (IPS) and queries (QPS). The range of values for IPS and QPS is split into 3 parts: bottom 25%, middle 50%, top 25%. Values in the bottom 25% have a red background, values in the top 25% have a green background and values in the middle have no color. A gray background is used for values that can be ignored because the DBMS did not sustain the target insert rate. Red backgrounds are not used when the minimum value is within 80% of the max value.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
20m.my5651_rel.cy8_1g 95238475188507280225352200
20m.my5740_rel_native_lto.cy8_1g 766281020305831248022951946
20m.my8022_rel_native_lto.cy9_1g 6250012539218468412598
20m.my8028_rel_native_lto.cy9_1g 62696197062111861146114
20m.my8031_rel_native_lto.cy9_1g 58480213602314885174126
20m.my8032_rel_native_lto.cy9_1g 58140213602326854174125

This table has relative throughput, throughput for the DBMS relative to the DBMS in the first line, using the absolute throughput from the previous table.

dbmsl.i0l.xl.i1q100.1q500.1q1000.1
20m.my5651_rel.cy8_1g 1.001.001.001.001.001.00
20m.my5740_rel_native_lto.cy8_1g 0.802.150.690.890.910.88
20m.my8022_rel_native_lto.cy9_1g 0.660.260.260.240.050.04
20m.my8028_rel_native_lto.cy9_1g 0.660.410.250.310.060.05
20m.my8031_rel_native_lto.cy9_1g 0.610.450.270.320.070.06
20m.my8032_rel_native_lto.cy9_1g 0.610.450.270.300.070.06

This lists the average rate of inserts/s for the tests that do inserts concurrent with queries. For such tests the query rate is listed in the table above. The read+write tests are setup so that the insert rate should match the target rate every second. Cells that are not at least 95% of the target have a red background to indicate a failure to satisfy the target.

dbmsq100.1q500.1q1000.1
my5651_rel.cy8_1g100499998
my5740_rel_native_lto.cy8_1g100499998
my8022_rel_native_lto.cy9_1g100378457
my8028_rel_native_lto.cy9_1g100472640
my8031_rel_native_lto.cy9_1g100496709
my8032_rel_native_lto.cy9_1g100495711
target1005001000

l.i0

l.i0: load without secondary indexes. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g88.17111.2020.6150.0090.0030.097
my5740_rel_native_lto.cy8_1g3.44994.8271.5720.1510.0010.066
my8022_rel_native_lto.cy9_1g99.5640.2430.1780.0150.104
my8028_rel_native_lto.cy9_1g99.4400.4150.1370.0090.104
my8031_rel_native_lto.cy9_1g99.3940.4710.1320.0030.123
my8032_rel_native_lto.cy9_1g99.3790.4500.1690.0020.127

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
95238	0	838	3.3	71.4	26.6	0.009	0.035	0.001	0.286	11187	21.7	0.117	18	1.3	41.8	1.0	0.097	98709	59635	20m.my5651_rel.cy8_1g
76628	0	0	0.0	103.3	21.5	0.000	0.000	0.001	0.287	7909	20.6	0.103	22	1.3	41.8	1.1	0.066	76559	67326	20m.my5740_rel_native_lto.cy8_1g
62500	0	0	0.0	161.4	18.3	0.000	0.000	0.003	0.299	7011	21.2	0.112	27	1.3	9.9	1.3	0.104	63029	54239	20m.my8022_rel_native_lto.cy9_1g
62696	0	0	0.0	165.0	18.4	0.000	0.000	0.003	0.300	7088	21.5	0.113	27	1.3	9.9	1.3	0.104	63412	53473	20m.my8028_rel_native_lto.cy9_1g
58480	0	0	0.0	172.4	17.8	0.000	0.000	0.003	0.311	6975	21.1	0.119	29	1.3	9.9	1.3	0.123	58933	49444	20m.my8031_rel_native_lto.cy9_1g
58140	0	0	0.0	173.2	17.8	0.000	0.000	0.003	0.313	6964	20.9	0.120	29	1.3	9.9	1.4	0.127	58734	44452	20m.my8032_rel_native_lto.cy9_1g

l.x

l.x: create secondary indexes.

Average throughput:

Image

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
47518	0	24	0.1	345.7	48.4	0.001	0.002	0.007	1.042	2089	6.3	0.044	11	2.7	43.2	1.1	0.012	NA	NA	20m.my5651_rel.cy8_1g
102030	0	0	0.0	342.1	91.7	0.000	0.000	0.003	0.921	4960	9.7	0.049	8	3.0	43.5	1.3	0.012	NA	NA	20m.my5740_rel_native_lto.cy8_1g
12539	0	0	0.0	317.8	11.2	0.000	0.000	0.025	0.914	2385	1.5	0.190	10	3.0	11.5	1.5	0.028	NA	NA	20m.my8022_rel_native_lto.cy9_1g
19706	0	102	6.8	543.4	22.2	0.005	0.355	0.028	1.154	4024	6.9	0.204	28	3.0	11.5	1.4	0.018	NA	NA	20m.my8028_rel_native_lto.cy9_1g
21360	0	110	7.4	575.6	23.7	0.005	0.354	0.027	1.137	4408	6.8	0.206	25	3.0	11.5	1.5	0.012	NA	NA	20m.my8031_rel_native_lto.cy9_1g
21360	0	110	7.4	574.4	23.7	0.005	0.354	0.027	1.137	4396	7.9	0.206	30	3.0	11.5	1.5	0.034	NA	NA	20m.my8032_rel_native_lto.cy9_1g

l.i1

l.i1: continue load after secondary indexes created. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For the max column values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g42.65052.8284.4600.0560.0050.871
my5740_rel_native_lto.cy8_1g2.02088.4219.4810.0720.0060.574
my8022_rel_native_lto.cy9_1g1.05360.66931.9985.9460.2890.046nonzero4.122
my8028_rel_native_lto.cy9_1g0.73758.76132.8657.4170.2080.012nonzero7.464
my8031_rel_native_lto.cy9_1g0.47260.39433.5785.4090.1220.0253.057
my8032_rel_native_lto.cy9_1g0.56760.55333.3555.3780.1230.0253.897

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
8507	0	154	0.6	1769.3	66.7	0.018	0.072	0.208	8.031	5908	14.9	0.695	140	7.7	48.2	1.1	0.871	8640	1000	20m.my5651_rel.cy8_1g
5831	0	0	0.0	1398.4	50.1	0.000	0.001	0.240	8.805	5025	13.9	0.862	191	7.7	48.3	1.3	0.574	5843	1398	20m.my5740_rel_native_lto.cy8_1g
2184	0	0	0.0	1063.6	21.4	0.000	0.000	0.487	10.032	5786	6.8	2.650	249	7.7	16.3	1.5	4.122	2347	50	20m.my8022_rel_native_lto.cy9_1g
2111	0	0	0.0	902.1	18.1	0.000	0.000	0.427	8.769	5066	6.9	2.400	262	7.7	16.3	1.5	7.464	2197	50	20m.my8028_rel_native_lto.cy9_1g
2314	0	0	0.0	1039.1	21.2	0.000	0.001	0.449	9.377	5875	8.0	2.539	277	7.7	16.3	1.5	3.057	2447	50	20m.my8031_rel_native_lto.cy9_1g
2326	0	0	0.0	1041.3	21.3	0.000	0.000	0.448	9.360	5887	8.0	2.531	275	7.7	16.3	1.5	3.897	2447	50	20m.my8032_rel_native_lto.cy9_1g

q100.1

q100.1: range queries with 100 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g34.69564.5980.2620.3990.0450.0010.190
my5740_rel_native_lto.cy8_1g19.70479.5370.1910.5650.0030.057
my8022_rel_native_lto.cy9_1g7.09488.0510.4462.1882.0060.2140.0010.387
my8028_rel_native_lto.cy9_1g6.38689.0370.4982.3651.6550.060nonzero0.259
my8031_rel_native_lto.cy9_1g3.38092.0250.6012.4181.5310.046nonzero0.272
my8032_rel_native_lto.cy9_1g1.59293.6650.6132.4531.6310.045nonzero0.271

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g28.63952.22218.5280.6110.254
my5740_rel_native_lto.cy8_1g17.69471.27810.1390.8890.121
my8022_rel_native_lto.cy9_1g0.80630.69412.86133.88921.7220.0281.056
my8028_rel_native_lto.cy9_1g1.47243.22211.94434.9178.4440.770
my8031_rel_native_lto.cy9_1g0.30645.75012.66734.3066.9720.843
my8032_rel_native_lto.cy9_1g0.13945.27812.86135.1676.5560.810

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
100	2802	3	0.0	477.0	14.5	0.001	0.007	4.779	148.339	11892	12.0	4.244	343	7.9	48.4	1.1	0.190	2861	911	20m.my5651_rel.cy8_1g
100	2480	1	0.0	483.2	14.5	0.000	0.006	4.841	148.704	10877	12.4	4.386	400	7.9	48.4	1.3	0.057	2525	784	20m.my5740_rel_native_lto.cy8_1g
100	684	1	0.0	633.7	11.9	0.002	0.028	6.349	121.732	6164	6.1	9.016	714	7.9	16.5	1.5	0.387	320	16	20m.my8022_rel_native_lto.cy9_1g
100	861	1	0.0	663.1	12.2	0.001	0.018	6.644	124.774	6903	6.8	8.021	632	7.9	16.5	1.5	0.259	655	32	20m.my8028_rel_native_lto.cy9_1g
100	885	1	0.0	673.4	12.4	0.001	0.016	6.748	126.866	7234	7.3	8.174	660	7.9	16.5	1.5	0.272	640	32	20m.my8031_rel_native_lto.cy9_1g
100	854	1	0.0	670.0	12.3	0.001	0.019	6.713	126.356	7094	7.3	8.309	684	7.9	16.5	1.5	0.271	703	32	20m.my8032_rel_native_lto.cy9_1g

q500.1

q500.1: range queries with 500 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g31.37667.7500.0860.6990.089nonzero0.089
my5740_rel_native_lto.cy8_1g17.83681.2810.0180.864nonzero0.025
my8022_rel_native_lto.cy9_1g1.22064.0811.04618.08513.8851.6820.0010.315
my8028_rel_native_lto.cy9_1g1.20362.4952.01219.85613.8260.6060.0020.389
my8031_rel_native_lto.cy9_1g0.65164.5142.42120.84011.3500.2250.223
my8032_rel_native_lto.cy9_1g0.39665.1652.40620.53611.2200.2770.227

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g52.60642.2005.1830.0110.073
my5740_rel_native_lto.cy8_1g34.49464.2111.2940.058
my8022_rel_native_lto.cy9_1g0.32821.29421.06144.98312.3330.759
my8028_rel_native_lto.cy9_1g0.48314.78324.26159.3061.1670.606
my8031_rel_native_lto.cy9_1g0.04417.81726.95054.9890.2000.592
my8032_rel_native_lto.cy9_1g0.06718.40627.51753.6170.3940.487

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
499	2535	14	0.1	1173.0	35.6	0.005	0.043	2.350	73.134	12377	11.9	4.882	376	8.3	48.8	1.1	0.089	2541	2349	20m.my5651_rel.cy8_1g
499	2295	5	0.1	1148.4	34.9	0.002	0.035	2.301	71.697	11715	12.7	5.104	443	8.3	48.9	1.3	0.025	2285	2141	20m.my5740_rel_native_lto.cy8_1g
378	125	3	0.1	775.8	14.7	0.027	0.464	2.055	39.769	5075	4.1	40.734	2633	8.3	16.9	1.5	0.315	48	16	20m.my8022_rel_native_lto.cy9_1g
472	146	5	0.1	908.8	17.3	0.032	0.546	1.925	37.490	5608	4.7	38.435	2577	8.3	16.9	1.5	0.389	80	32	20m.my8028_rel_native_lto.cy9_1g
496	174	4	0.1	1012.6	19.3	0.024	0.412	2.040	39.820	6383	5.6	36.684	2575	8.3	16.9	1.5	0.223	96	48	20m.my8031_rel_native_lto.cy9_1g
495	174	5	0.1	1012.1	19.3	0.031	0.440	2.045	39.920	6361	5.6	36.665	2582	8.3	16.9	1.5	0.227	96	32	20m.my8032_rel_native_lto.cy9_1g

q1000.1

q1000.1: range queries with 1000 insert/s per client. Graphs for performance per 1-second interval are here.

Average throughput:

Image

Query response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g27.57271.0940.0811.1360.117nonzero0.123
my5740_rel_native_lto.cy8_1g16.91981.6540.0451.2920.0860.0040.129
my8022_rel_native_lto.cy9_1g0.45459.0801.90119.24316.5402.7800.0030.373
my8028_rel_native_lto.cy9_1g0.28555.5403.31721.96917.8301.0570.0010.347
my8031_rel_native_lto.cy9_1g0.14054.4175.02723.83015.9720.615nonzero0.279
my8032_rel_native_lto.cy9_1g0.05354.3175.13323.63516.2500.6100.0010.339

Insert response time histogram: each cell has the percentage of responses that take <= the time in the header and max is the max response time in seconds. For max values in the top 25% of the range have a red background and in the bottom 25% of the range have a green background. The red background is not used when the min value is within 80% of the max value.

dbms256us1ms4ms16ms64ms256ms1s4s16sgtmax
my5651_rel.cy8_1g50.38644.5535.0360.0250.223
my5740_rel_native_lto.cy8_1g14.85882.4222.4000.3190.165
my8022_rel_native_lto.cy9_1g0.07531.04716.86941.93910.0690.880
my8028_rel_native_lto.cy9_1g0.06121.93327.43149.8220.7530.648
my8031_rel_native_lto.cy9_1g0.00822.65630.99746.1440.1940.424
my8032_rel_native_lto.cy9_1g0.02223.29730.63345.8890.1580.580

Performance metrics for the DBMS listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
998	2200	25	0.2	1988.3	60.6	0.011	0.084	1.992	62.211	12684	12.2	5.767	444	8.8	49.3	1.1	0.123	2205	1933	20m.my5651_rel.cy8_1g
998	1946	8	0.1	1921.7	58.5	0.004	0.071	1.925	60.017	12622	14.0	6.486	576	8.8	49.3	1.3	0.129	2014	1183	20m.my5740_rel_native_lto.cy8_1g
457	98	2	0.0	751.9	14.3	0.018	0.295	1.644	31.924	4878	4.0	49.722	3262	8.8	17.3	1.5	0.373	48	16	20m.my8022_rel_native_lto.cy9_1g
640	114	3	0.0	923.2	17.7	0.024	0.383	1.441	28.232	5551	5.0	48.864	3521	8.8	17.3	1.5	0.347	64	16	20m.my8028_rel_native_lto.cy9_1g
709	126	2	0.0	1024.6	19.7	0.020	0.322	1.444	28.488	6265	5.9	49.642	3740	8.8	17.3	1.5	0.279	80	32	20m.my8031_rel_native_lto.cy9_1g
711	125	3	0.0	1021.2	19.7	0.025	0.367	1.437	28.369	6238	5.8	49.820	3706	8.8	17.3	1.5	0.339	80	32	20m.my8032_rel_native_lto.cy9_1g

l.i0

l.i0: load without secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
95238	0	838	3.3	71.4	26.6	0.009	0.035	0.001	0.286	11187	21.7	0.117	18	1.3	41.8	1.0	0.097	98709	59635	20m.my5651_rel.cy8_1g
76628	0	0	0.0	103.3	21.5	0.000	0.000	0.001	0.287	7909	20.6	0.103	22	1.3	41.8	1.1	0.066	76559	67326	20m.my5740_rel_native_lto.cy8_1g
62500	0	0	0.0	161.4	18.3	0.000	0.000	0.003	0.299	7011	21.2	0.112	27	1.3	9.9	1.3	0.104	63029	54239	20m.my8022_rel_native_lto.cy9_1g
62696	0	0	0.0	165.0	18.4	0.000	0.000	0.003	0.300	7088	21.5	0.113	27	1.3	9.9	1.3	0.104	63412	53473	20m.my8028_rel_native_lto.cy9_1g
58480	0	0	0.0	172.4	17.8	0.000	0.000	0.003	0.311	6975	21.1	0.119	29	1.3	9.9	1.3	0.123	58933	49444	20m.my8031_rel_native_lto.cy9_1g
58140	0	0	0.0	173.2	17.8	0.000	0.000	0.003	0.313	6964	20.9	0.120	29	1.3	9.9	1.4	0.127	58734	44452	20m.my8032_rel_native_lto.cy9_1g

l.x

l.x: create secondary indexes

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
47518	0	24	0.1	345.7	48.4	0.001	0.002	0.007	1.042	2089	6.3	0.044	11	2.7	43.2	1.1	0.012	NA	NA	20m.my5651_rel.cy8_1g
102030	0	0	0.0	342.1	91.7	0.000	0.000	0.003	0.921	4960	9.7	0.049	8	3.0	43.5	1.3	0.012	NA	NA	20m.my5740_rel_native_lto.cy8_1g
12539	0	0	0.0	317.8	11.2	0.000	0.000	0.025	0.914	2385	1.5	0.190	10	3.0	11.5	1.5	0.028	NA	NA	20m.my8022_rel_native_lto.cy9_1g
19706	0	102	6.8	543.4	22.2	0.005	0.355	0.028	1.154	4024	6.9	0.204	28	3.0	11.5	1.4	0.018	NA	NA	20m.my8028_rel_native_lto.cy9_1g
21360	0	110	7.4	575.6	23.7	0.005	0.354	0.027	1.137	4408	6.8	0.206	25	3.0	11.5	1.5	0.012	NA	NA	20m.my8031_rel_native_lto.cy9_1g
21360	0	110	7.4	574.4	23.7	0.005	0.354	0.027	1.137	4396	7.9	0.206	30	3.0	11.5	1.5	0.034	NA	NA	20m.my8032_rel_native_lto.cy9_1g

l.i1

l.i1: continue load after secondary indexes created

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
8507	0	154	0.6	1769.3	66.7	0.018	0.072	0.208	8.031	5908	14.9	0.695	140	7.7	48.2	1.1	0.871	8640	1000	20m.my5651_rel.cy8_1g
5831	0	0	0.0	1398.4	50.1	0.000	0.001	0.240	8.805	5025	13.9	0.862	191	7.7	48.3	1.3	0.574	5843	1398	20m.my5740_rel_native_lto.cy8_1g
2184	0	0	0.0	1063.6	21.4	0.000	0.000	0.487	10.032	5786	6.8	2.650	249	7.7	16.3	1.5	4.122	2347	50	20m.my8022_rel_native_lto.cy9_1g
2111	0	0	0.0	902.1	18.1	0.000	0.000	0.427	8.769	5066	6.9	2.400	262	7.7	16.3	1.5	7.464	2197	50	20m.my8028_rel_native_lto.cy9_1g
2314	0	0	0.0	1039.1	21.2	0.000	0.001	0.449	9.377	5875	8.0	2.539	277	7.7	16.3	1.5	3.057	2447	50	20m.my8031_rel_native_lto.cy9_1g
2326	0	0	0.0	1041.3	21.3	0.000	0.000	0.448	9.360	5887	8.0	2.531	275	7.7	16.3	1.5	3.897	2447	50	20m.my8032_rel_native_lto.cy9_1g

q100.1

q100.1: range queries with 100 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
100	2802	3	0.0	477.0	14.5	0.001	0.007	4.779	148.339	11892	12.0	4.244	343	7.9	48.4	1.1	0.190	2861	911	20m.my5651_rel.cy8_1g
100	2480	1	0.0	483.2	14.5	0.000	0.006	4.841	148.704	10877	12.4	4.386	400	7.9	48.4	1.3	0.057	2525	784	20m.my5740_rel_native_lto.cy8_1g
100	684	1	0.0	633.7	11.9	0.002	0.028	6.349	121.732	6164	6.1	9.016	714	7.9	16.5	1.5	0.387	320	16	20m.my8022_rel_native_lto.cy9_1g
100	861	1	0.0	663.1	12.2	0.001	0.018	6.644	124.774	6903	6.8	8.021	632	7.9	16.5	1.5	0.259	655	32	20m.my8028_rel_native_lto.cy9_1g
100	885	1	0.0	673.4	12.4	0.001	0.016	6.748	126.866	7234	7.3	8.174	660	7.9	16.5	1.5	0.272	640	32	20m.my8031_rel_native_lto.cy9_1g
100	854	1	0.0	670.0	12.3	0.001	0.019	6.713	126.356	7094	7.3	8.309	684	7.9	16.5	1.5	0.271	703	32	20m.my8032_rel_native_lto.cy9_1g

q500.1

q500.1: range queries with 500 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
499	2535	14	0.1	1173.0	35.6	0.005	0.043	2.350	73.134	12377	11.9	4.882	376	8.3	48.8	1.1	0.089	2541	2349	20m.my5651_rel.cy8_1g
499	2295	5	0.1	1148.4	34.9	0.002	0.035	2.301	71.697	11715	12.7	5.104	443	8.3	48.9	1.3	0.025	2285	2141	20m.my5740_rel_native_lto.cy8_1g
378	125	3	0.1	775.8	14.7	0.027	0.464	2.055	39.769	5075	4.1	40.734	2633	8.3	16.9	1.5	0.315	48	16	20m.my8022_rel_native_lto.cy9_1g
472	146	5	0.1	908.8	17.3	0.032	0.546	1.925	37.490	5608	4.7	38.435	2577	8.3	16.9	1.5	0.389	80	32	20m.my8028_rel_native_lto.cy9_1g
496	174	4	0.1	1012.6	19.3	0.024	0.412	2.040	39.820	6383	5.6	36.684	2575	8.3	16.9	1.5	0.223	96	48	20m.my8031_rel_native_lto.cy9_1g
495	174	5	0.1	1012.1	19.3	0.031	0.440	2.045	39.920	6361	5.6	36.665	2582	8.3	16.9	1.5	0.227	96	32	20m.my8032_rel_native_lto.cy9_1g

q1000.1

q1000.1: range queries with 1000 insert/s per client

Performance metrics for all DBMS, not just the ones listed above. Some are normalized by throughput, others are not. Legend for results is here.

ips	qps	rps	rmbps	wps	wmbps	rpq	rkbpq	wpi	wkbpi	csps	cpups	cspq	cpupq	dbgb1	dbgb2	rss	maxop	p50	p99	tag
998	2200	25	0.2	1988.3	60.6	0.011	0.084	1.992	62.211	12684	12.2	5.767	444	8.8	49.3	1.1	0.123	2205	1933	20m.my5651_rel.cy8_1g
998	1946	8	0.1	1921.7	58.5	0.004	0.071	1.925	60.017	12622	14.0	6.486	576	8.8	49.3	1.3	0.129	2014	1183	20m.my5740_rel_native_lto.cy8_1g
457	98	2	0.0	751.9	14.3	0.018	0.295	1.644	31.924	4878	4.0	49.722	3262	8.8	17.3	1.5	0.373	48	16	20m.my8022_rel_native_lto.cy9_1g
640	114	3	0.0	923.2	17.7	0.024	0.383	1.441	28.232	5551	5.0	48.864	3521	8.8	17.3	1.5	0.347	64	16	20m.my8028_rel_native_lto.cy9_1g
709	126	2	0.0	1024.6	19.7	0.020	0.322	1.444	28.488	6265	5.9	49.642	3740	8.8	17.3	1.5	0.279	80	32	20m.my8031_rel_native_lto.cy9_1g
711	125	3	0.0	1021.2	19.7	0.025	0.367	1.437	28.369	6238	5.8	49.820	3706	8.8	17.3	1.5	0.339	80	32	20m.my8032_rel_native_lto.cy9_1g

l.i0

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	88.171	11.202	0.615	0.009	0.003	0.000	0.000	0.000	0.000	0.097	my5651_rel.cy8_1g
0.000	3.449	94.827	1.572	0.151	0.001	0.000	0.000	0.000	0.000	0.066	my5740_rel_native_lto.cy8_1g
0.000	0.000	99.564	0.243	0.178	0.015	0.000	0.000	0.000	0.000	0.104	my8022_rel_native_lto.cy9_1g
0.000	0.000	99.440	0.415	0.137	0.009	0.000	0.000	0.000	0.000	0.104	my8028_rel_native_lto.cy9_1g
0.000	0.000	99.394	0.471	0.132	0.003	0.000	0.000	0.000	0.000	0.123	my8031_rel_native_lto.cy9_1g
0.000	0.000	99.379	0.450	0.169	0.002	0.000	0.000	0.000	0.000	0.127	my8032_rel_native_lto.cy9_1g

l.x

TODO - determine whether there is data for create index response time


l.i1

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	42.650	52.828	4.460	0.056	0.005	0.000	0.000	0.000	0.871	my5651_rel.cy8_1g
0.000	0.000	2.020	88.421	9.481	0.072	0.006	0.000	0.000	0.000	0.574	my5740_rel_native_lto.cy8_1g
0.000	0.000	1.053	60.669	31.998	5.946	0.289	0.046	nonzero	0.000	4.122	my8022_rel_native_lto.cy9_1g
0.000	0.000	0.737	58.761	32.865	7.417	0.208	0.012	nonzero	0.000	7.464	my8028_rel_native_lto.cy9_1g
0.000	0.000	0.472	60.394	33.578	5.409	0.122	0.025	0.000	0.000	3.057	my8031_rel_native_lto.cy9_1g
0.000	0.000	0.567	60.553	33.355	5.378	0.123	0.025	0.000	0.000	3.897	my8032_rel_native_lto.cy9_1g

q100.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
34.695	64.598	0.262	0.399	0.045	0.001	0.000	0.000	0.000	0.000	0.190	my5651_rel.cy8_1g
19.704	79.537	0.191	0.565	0.003	0.000	0.000	0.000	0.000	0.000	0.057	my5740_rel_native_lto.cy8_1g
7.094	88.051	0.446	2.188	2.006	0.214	0.001	0.000	0.000	0.000	0.387	my8022_rel_native_lto.cy9_1g
6.386	89.037	0.498	2.365	1.655	0.060	nonzero	0.000	0.000	0.000	0.259	my8028_rel_native_lto.cy9_1g
3.380	92.025	0.601	2.418	1.531	0.046	nonzero	0.000	0.000	0.000	0.272	my8031_rel_native_lto.cy9_1g
1.592	93.665	0.613	2.453	1.631	0.045	nonzero	0.000	0.000	0.000	0.271	my8032_rel_native_lto.cy9_1g

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	28.639	52.222	18.528	0.611	0.000	0.000	0.000	0.000	0.254	my5651_rel.cy8_1g
0.000	0.000	17.694	71.278	10.139	0.889	0.000	0.000	0.000	0.000	0.121	my5740_rel_native_lto.cy8_1g
0.000	0.000	0.806	30.694	12.861	33.889	21.722	0.028	0.000	0.000	1.056	my8022_rel_native_lto.cy9_1g
0.000	0.000	1.472	43.222	11.944	34.917	8.444	0.000	0.000	0.000	0.770	my8028_rel_native_lto.cy9_1g
0.000	0.000	0.306	45.750	12.667	34.306	6.972	0.000	0.000	0.000	0.843	my8031_rel_native_lto.cy9_1g
0.000	0.000	0.139	45.278	12.861	35.167	6.556	0.000	0.000	0.000	0.810	my8032_rel_native_lto.cy9_1g

q500.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
31.376	67.750	0.086	0.699	0.089	nonzero	0.000	0.000	0.000	0.000	0.089	my5651_rel.cy8_1g
17.836	81.281	0.018	0.864	nonzero	0.000	0.000	0.000	0.000	0.000	0.025	my5740_rel_native_lto.cy8_1g
1.220	64.081	1.046	18.085	13.885	1.682	0.001	0.000	0.000	0.000	0.315	my8022_rel_native_lto.cy9_1g
1.203	62.495	2.012	19.856	13.826	0.606	0.002	0.000	0.000	0.000	0.389	my8028_rel_native_lto.cy9_1g
0.651	64.514	2.421	20.840	11.350	0.225	0.000	0.000	0.000	0.000	0.223	my8031_rel_native_lto.cy9_1g
0.396	65.165	2.406	20.536	11.220	0.277	0.000	0.000	0.000	0.000	0.227	my8032_rel_native_lto.cy9_1g

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	52.606	42.200	5.183	0.011	0.000	0.000	0.000	0.000	0.073	my5651_rel.cy8_1g
0.000	0.000	34.494	64.211	1.294	0.000	0.000	0.000	0.000	0.000	0.058	my5740_rel_native_lto.cy8_1g
0.000	0.000	0.328	21.294	21.061	44.983	12.333	0.000	0.000	0.000	0.759	my8022_rel_native_lto.cy9_1g
0.000	0.000	0.483	14.783	24.261	59.306	1.167	0.000	0.000	0.000	0.606	my8028_rel_native_lto.cy9_1g
0.000	0.000	0.044	17.817	26.950	54.989	0.200	0.000	0.000	0.000	0.592	my8031_rel_native_lto.cy9_1g
0.000	0.000	0.067	18.406	27.517	53.617	0.394	0.000	0.000	0.000	0.487	my8032_rel_native_lto.cy9_1g

q1000.1

Query response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
27.572	71.094	0.081	1.136	0.117	nonzero	0.000	0.000	0.000	0.000	0.123	my5651_rel.cy8_1g
16.919	81.654	0.045	1.292	0.086	0.004	0.000	0.000	0.000	0.000	0.129	my5740_rel_native_lto.cy8_1g
0.454	59.080	1.901	19.243	16.540	2.780	0.003	0.000	0.000	0.000	0.373	my8022_rel_native_lto.cy9_1g
0.285	55.540	3.317	21.969	17.830	1.057	0.001	0.000	0.000	0.000	0.347	my8028_rel_native_lto.cy9_1g
0.140	54.417	5.027	23.830	15.972	0.615	nonzero	0.000	0.000	0.000	0.279	my8031_rel_native_lto.cy9_1g
0.053	54.317	5.133	23.635	16.250	0.610	0.001	0.000	0.000	0.000	0.339	my8032_rel_native_lto.cy9_1g

Insert response time histogram

256us	1ms	4ms	16ms	64ms	256ms	1s	4s	16s	gt	max	tag
0.000	0.000	50.386	44.553	5.036	0.025	0.000	0.000	0.000	0.000	0.223	my5651_rel.cy8_1g
0.000	0.000	14.858	82.422	2.400	0.319	0.000	0.000	0.000	0.000	0.165	my5740_rel_native_lto.cy8_1g
0.000	0.000	0.075	31.047	16.869	41.939	10.069	0.000	0.000	0.000	0.880	my8022_rel_native_lto.cy9_1g
0.000	0.000	0.061	21.933	27.431	49.822	0.753	0.000	0.000	0.000	0.648	my8028_rel_native_lto.cy9_1g
0.000	0.000	0.008	22.656	30.997	46.144	0.194	0.000	0.000	0.000	0.424	my8031_rel_native_lto.cy9_1g
0.000	0.000	0.022	23.297	30.633	45.889	0.158	0.000	0.000	0.000	0.580	my8032_rel_native_lto.cy9_1g