\c 25 200i / c: count | ht: historical table | st: snapshot table | rt: realtime table | tx: transaction generateRandomHdbFrom:{[c;u]flip`e`v`t!(c?c*4;u@c?count u;1+til c)} generateRandomHdb:generateRandomHdbFrom[;"abcdefghijklmnopqrstuvwxyz"] appendToHdbFrom:{[ht;c;u]ht upsert flip(c?100000*count u;c?u;(1+$[0~count lastTx:select [-1] t from ht;0;first lastTx@`t]))} appendToHdb:appendToHdbFrom[;;"abcdefghijklmnopqrstuvwxyz"] createSnapshotTable:{[ht;tx]tx:$[-1~tx;0Wj;tx];s:`v xasc select e,v from ht where t <= tx;w:last each value group s@`e;`e xkey s@w} / input: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 / output: 676 169 81 49 36 25 16 16 9 9 9 9 4 4 4 4 4 4 4 4 4 4 4 4 4 1 generateRandomTableHandles:{[crossProductAlphabetSize]`$(":db/",/:raze T,\:/:T:cut[crossProductAlphabetSize;"abcdefghijklmnopqrstuvwxyz"]),'"/"} resetHdbTables:{[th]set[;([] e:`long$(); v:`char$(); t:`long$())] each th} resetHdb:{} / RUN / appendToTable[;1000000] each th / appendToTable[;10] each th / RESULTS in MBP with 16 GB, 2.6 GHz Intel Core i7 and SSD: / 169 Tables / 169 million rows appended in 22 seg. total ~2.87 GB (169 tables, 1 million each) / after that, 10 rows are appended to all 169 tables in 50-200 ms / 25 Tables / 25 million rows appended in 3.7 seg. total ~425MB (25 tables, 1 million each) / after that, 10 rows are appended to all 25 tables in ~25ms / 9 Tables / 9 million rows appended in 1.5 seg. total ~153MB (9 tables, 1 million each) / after that, 10 rows are appended to all 9 tables in ~8ms / 4 Tables / 4 million rows appended in 0.6 seg. total ~68MB (4 tables, 1 million each) / after that, 10 rows are appended to all 4 tables in ~4ms / Throughput : 10-250 writes per second w/o transaction log