elasticsearch - Data Store with multiple shard keys -


i've been researching several different data store technologies can used storing huge amounts of semi-structured logs people search through later. i've looked @ cassandra, riak, , elastic search far, , seems elastic search offers closest fit i'm interested in (largely because indexes transparently). however, there's 1 feature i'm interested in seems escape them all, , wondering if there data store feature.

what i'm thinking ability transparent shard on multiple key. clear, i'm not talking using composite key sharding. mean if had table sharded user_id, time_of_creation, , ip_address, , inserted row, 3 copies of row created, each 1 in different cluster that's sharded different key (or maybe somehow in same cluster. important part data duplicated). , when wanted query table later, data store transparently choose cluster use.

in articles i've read cassandra, people recommend doing this, it's manual process in @ least 3 ways:

  • for insertion, have insert each table yourself.
  • when comes querying, have figure out table want query (you need pick 1 uses right cluster key).
  • and, if ever want add key shard on, have write routine existing data new table.

although using cassandra example, believe situation riak , elastic search similar. understand data store offered ability have make huge trades so. updating/deleting might no longer possible (or have extremely poor performance), , consistency suffer. but, it's set of trades find acceptable when dealing logs, wondering if familiar technology offers feature.


Comments

Popular posts from this blog

javascript - Slick Slider width recalculation -

jsf - PrimeFaces Datatable - What is f:facet actually doing? -

angular2 services - Angular 2 RC 4 Http post not firing -