The network does not use dedicated servers to maintain the metadata information, it supports redundant objects storage. Small to medium sized write benchmarks can be found on eblob page.
Distributed hash table design allows not to use dedicated metadata servers which frequently become points of failure in the classical storages, instead user can connect to any server in the network and all requests will be forwarded to the needed nodes. One can lookup the needed server and connect there directly to get data.
Elliptics can be called a cloud of losely connected equivalent nodes. Joining node will automatically connect to the needed servers according to the network topology, it can store data in different configurable backends like file IO storage, eblob backend. One can create own IO storage backend.
Protocol allows to implement own data storage using specific features for the deploying project and generally extend data communication with infinite number of the extensions. One of the implemented examples is remote command execution, which can be used as a load balancing job manager.
Short feature list includes:
- Distributed hash tables, no metadata servers, true horizontal scaling
- Data replication – replicas can be stored in different physical locations and no Amazon-like outages are ever possible
- Column data storage (eblob only)
- Range requests
- Different IO storage backends, modular architecture which allows to easily implement own transaction storage
- Automatic data repartitioning in case of removed or added nodes
- Ring addressing structure, ability to implement own key generation models
- Cluster statistics gathering
- IO notifications support for any object in the network
- Automatic configuration
- HTTP frontend (fastcgi application, benchmark), C/C++ and Python bindings, async, bulk and range operations
- Google’s Snappy compression support (eblob)
- Extremely fast LevelDB backend for small compressable records
- Server-side atomic execution support
- Seconday indexes support
We started www.elliptics.ru community support site to gather feedback and information.
Linear scalability of the communication channel breaks the limits of the horizontal server extensions which if used with the replication facilities allows to implement any level of the data redundancy.
Elliptics technology currenly hosts a major set of services of Yandex search engine, like music, photos, mail backup and many others.