Data recovery in Elliptics

As you know, Elliptics includes built-in utility for data synchronization both within one replica and between number of replicas. In this article I describe different modes in which synchronization could be run, how it works and give details on situations when it could be used.

dnet_recovery is the main utility for data synchronization in Elliptics. It is distributed with the package elliptics-client. dnet_recovery supports 2 modes: `merge` and `dc`. `merge` is intended for data synchronization within one replica and `dc` – for data synchronization between number of replicas. Each of the modes can automatically search keys for synchronization or use keys from dump file.

Most of described bellow can be found at recovery doxygen documentation. All doxygen documentation will be published on soon.

Continue reading

Recent eblob wiki updates

Hi, it’s SaveTheRbtz speaking, I’m an engineer responsible to eblob code maintenance and improvement. There have been much of work done with eblob code base for past half a year, including big stuff like data-sort and l2lash, but also there were tons of small but useful improvements like rwlocks, memory usage improvements and start up speed.

I’m here to present recent wiki updates regarding some eblob features e.g. one that explains how reading and writing in eblob works and what optimizations make it faster than plain directory with billions of files.

Also we’ve expanded data-sort manual: now it has more info about data-sort and defragmentation including pros and cons.

Next release of eblob will break backward compatibility in terms of both ABI and API, but it will allow us not only improve performance and provide new features but also greatly simplify API itself and remove some useless functionality. For example we are now working on writev()-like (aka scatter/gather) interface for blob that will allow us to efficiently implement metadata in elliptics.

Full roadmap for eblob could be seen in corresponding wiki article too.