K-V DB

####1. K-V DB Key-value (KV) stores use the associative array (also known as a map or dictionary) as their fundamental data model. In this model, data is represented as a collection of key-value pairs, such that each possible key appears at most once in the collection.

The key-value model is one of the simplest non-trivial data models, and richer data models are often implemented on top of it. The key-value model can be extended to an ordered model that maintains keys in lexicographic order. This extension is powerful, in that it can efficiently process key ranges.

Key-value stores can use consistency models ranging from eventual consistency to serializability. Some support ordering of keys. Some maintain data in memory (RAM), while others employ solid-state drives or rotating disks. Here is a list of key-value stores:

key-value includes following types:

  1. KV-eventually consistent
    • dynamo
    • riak
  2. KV-ordered
    • Berkeley DB
    • MemcacheDB
    • FoundationDB
  3. KV - RAM
    • Redis
    • memcached
    • ssdb
    • levelDB
  4. KV - solid-state drive or rotating disk
    • MongoDB
    • Oracle NoSQL Database
    • LevelDB
    • BigTable

####2. About Redis

Redis is an open source, BSD licensed, advanced key-value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs.

In order to achieve its outstanding performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log.Persistence can be optionally disabled, if you just need a feature-rich, networked, istent[edit]

Redis also supports trivial-to-setup master-slave asynchronous replication, with very fast non-blocking first synchronization, auto-reconnection with partial resynchronization on net split.

Features includes:

  1. Transactions
  2. Pub/Sub
  3. Keys with a limited time-to-live (Session Control)
  4. Automatic failover
  5. LRU eviction of keys

####3. When to use Redis?

If you can map a use case to Redis and discover you aren’t at risk of running out of RAM by using Redis there is a good chance you should probably use Redis. It’s a “NoSQL” key-value data store. More precisely, it is a data structure server.Not like MongoDB (which is a disk-based document store), though MongoDB could be used for similar key/value use cases

  1. Memcached. but with built-in persistence (snapshotting or journaling to disk) and more datatypes.Persistence to disk means you can use Redis as a real database instead of just a volatile cache. The data won’t disappear when you restart, like with memcached.

  2. data types. key values can be simple strings, like you’ll find in memcached, but they can also be more complex types like Hashes, Lists (ordered collection, makes a great queue), Sets (unordered collection of non-repeating values), or Sorted Sets (ordered/ranked collection of non-repeating values)

  3. vitual memory.
  4. scalable data store shared by multiple processes. As just an inter-process communication mechanism it is tough to beat. The fact that you can communicate cross-platform, cross-server, or cross-application just as easily makes it a pretty great choice for many many use cases. Its speed also makes it great as a caching layer.

####4. Detailed Applications:

  1. Show latest items listings in your home page. This is a live in-memory cache and is very fast. LPUSH is used to insert a content ID at the head of the list stored at a key. LTRIM is used to limit the number of items in the list to 5000. If the user needs to page beyond this cache only then are they sent to the database.

  2. Deletion and filtering. If a cached article is deleted it can be removed from the cache using LREM.

  3. Leaderboards and related problems. A leader board is a set sorted by score. The ZADD commands implements this directly and the ZREVRANGE command can be used to get the top 100 users by score and ZRANK can be used to get a users rank. Very direct and easy.

  4. Order by user votes and time. This is a leaderboard like Reddit where the score is formula the changes over time. LPUSH + LTRIM are used to add an article to a list. A background task polls the list and recomputes the order of the list and ZADD is used to populate the list in the new order. This list can be retrieved very fast by even a heavily loaded site. This should be easier, the need for the polling code isn’t elegant.

  5. Implement expires on items. To keep a sorted list by time then use unix time as the key. The difficult task of expiring items is implemented by indexing current_time+time_to_live. Another background worker is used to make queries using ZRANGE … with SCORES and delete timed out entries.

  6. Counting stuff. Keeping stats of all kinds is common, say you want to know when to block an IP addresss. The INCRBY command makes it easy to atomically keep counters; GETSET to atomically clear the counter; the expire attribute can be used to tell when an key should be deleted.

  7. Unique N items in a given amount of time. This is the unique visitors problem and can be solved using SADD for each pageview. SADD won’t add a member to a set if it already exists.

  8. Real time analysis of what is happening, for stats, anti spam, or whatever. Using Redis primitives it’s much simpler to implement a spam filtering system or other real-time tracking system.

  9. Pub/Sub. Keeping a map of who is interested in updates to what data is a common task in systems. Redis has a pub/sub feature to make this :easy using commands like SUBSCRIBE, UNSUBSCRIBE, and PUBLISH.

  10. Queues. Queues are everywhere in programming. In addition to the push and pop type commands, Redis has blocking queue commands so a program can wait on work being added to the queue by another program. You can also do interesting things implement a rotating queue of RSS feeds to update.

  11. Caching. Redis can be used in the same manner as memcache.

#####READ-LISTS

  1. How to take advantage of Redis just adding it to your stack
  2. 微博关系服务与Redis的故事

####4. Configuration Details

#####Password Settings

#edit in /etc/redis/redis.conf
requirepass 123456
$sudo service redis-server restart
$redis-cli -a 123456

#####Master-Slave Settings

#modify /etc/redis/redis-slave.conf(need to touch)
port 6378
slaveof 192.168.1.1 6379 #link to master ip and port
masterauth 123456 #master auto password, should be master password

Published: June 15 2015