dpiatek · March 22, 2016 11:59
diff --git a/dynamo.txt b/dynamo.txt
 Data considerations:
 Our recent non-peak throughput per month is about 60k requests, and much more in peak. An average order might be 3-4KB,
 so our monthly space use should be around 240MB. The main use case for almost all data is get/set and much less retrieving
 sets, which are only needed for the UI. The sets need to support ordering as well as filtering. 

 Design:
 Considering our relatively small workload and Dynamos constraints, we can put our "listable" data (orders, products,
 customers, addresses) into an single table where the hashkey is the type of data and sort key the id. Local indexes
 should be created for the sorting options we require at time of table creation. Local indexes reuse the table capacity
 so incur a lower cost, although they limit a given partitions size to 10GB, as well as actually copying the data thus
 incresing the size itself - this can be reduced by projecting (copying) only the attributes that we need for our
 results. 

 We can cicumvent the size problems by using a composite hash key, composed of the type of data + date truncated to the current month. 
 This would add some complexity in the application layer but would make the data more uniform, making the queries more efficient, 
 while allowing Dynamo to partion accordingly.

 Concering dates - they can be kept as ISO 8601 as they are sortable.
 Sortable attributes - all attributes that we require sorting by will need to be at top level.
	Data considerations:
	Our recent non-peak throughput per month is about 60k requests, and much more in peak. An average order might be 3-4KB,
	so our monthly space use should be around 240MB. The main use case for almost all data is get/set and much less retrieving
	sets, which are only needed for the UI. The sets need to support ordering as well as filtering.

	Design:
	Considering our relatively small workload and Dynamos constraints, we can put our "listable" data (orders, products,
	customers, addresses) into an single table where the hashkey is the type of data and sort key the id. Local indexes
	should be created for the sorting options we require at time of table creation. Local indexes reuse the table capacity
	so incur a lower cost, although they limit a given partitions size to 10GB, as well as actually copying the data thus
	incresing the size itself - this can be reduced by projecting (copying) only the attributes that we need for our
	results.

	We can cicumvent the size problems by using a composite hash key, composed of the type of data + date truncated to the current month.
	This would add some complexity in the application layer but would make the data more uniform, making the queries more efficient,
	while allowing Dynamo to partion accordingly.

	Concering dates - they can be kept as ISO 8601 as they are sortable.
	Sortable attributes - all attributes that we require sorting by will need to be at top level.
No results found