#lru cache

An LRU (least recently used) cache is a cache that saves memory by only storing the n most used items. 

It has two operations: get, and put that both run in constant time.

This is implemented using a combination of a hash table and a doubly linked list, in which the keys in the hash table point to individual nodes on the linked list.

The head of the linked list is considered the least recently used while the tail is considered the most recently used.

The get operation takes a key, and if it exists in the hash table, returns the node on the linked list. It then removes that node from the linked list and then re-attaches it to the tail.

Leetcode – LRU cache solution in Java

Design and implement a data structure for Least Recently Used (LRU) cache. It should support the following operations: get and set. get(key) – Get the value (will always be positive) of the key if the key exists in the cache, otherwise return -1. set(key, value) – Set or insert the value if the key is not already present. When the cache reached its capacity, it should invalidate the least…

Topics

Sets

Maps

Hashing functions

HashSets

HashMaps

LinkedLists

Singley-linked list

Doubley-linked list

LRU Cache

Big-Oh Space/Time Complexity

Projects

MaxIntSet : [0,2,3,5] --> [true, false, true, true, false, true]

O(1) lookup time, wasteful space

IntSet : fixed length array, use el%num_buckets to determine which bucket el will fall under

O(1) insertion, O(n) retrieval

ResizingIntSet : Double length of underlying fixed-length array when num_elements >= length

O(1) insertion, O(1) amortized retrieval

Hashing - built custom Array#hash,String#hash, Hashmp.hash using Ruby's Fixnum#hash method

HashSet - like previous Set, but use el.hash % num_buckets rather than el. Can now accept any type of element

LinkedList

Singly-linked - still take O(n) time to reach last object

Doubly-linked - more efficient

Remove links by reassigning points for @prev & @next to skip over current link

HashMap - use a Map with an underlying Linked List for the buckets

LRU Cache - use a Map

Least Recentuly Used - method of deciding what to keep/discard in memory

Keys -- hashed version of the original key

Values -- Link in ordered LinkedList

Keep length of linked list

Today I Learned: about hash maps, Linked lists, and LRU caches.

We had a brief Q&A regarding the Blackjack practice assessment to begin the day. Afterwards, we discussed various ways to optimize for time complexity by using hash maps and linked lists in an LRU cache. After the discussion, we separated into our pairs for the programming exercises for the day.

Hot on the heels of learning about big O notation, Friday’s task was to first take in about 3 hours of lecture via online videos, and then get to work designing a LRU Cache from scratch.  The goal was to design a data structure which can store, lookup, and delete elements all in constant time regardless of the size of the data set, and can store recently used items in RAM to make rendering more efficient.  We did this by first making our own versions of a linked list, an ordered list of items in memory that can be easily modified without changing the reference path to each of the objects in the list, unlike other data structures such as an array (in an array, objects can be referenced by their position in the array, but if I take something out or add something, everything’s position changes, whereas a linked list will maintain its relationships regardless of where I insert an element). We then designed a hash map (ruby’s built in Hash class, often notated with curly braces { }), which can be designed through a clever combination of an array of variable size (size increases as more things are stored in it, so that lookup times stay constant) in which each element is a linked list of stored objects.  Objects are stored by assigning them a unique but consistent hash number, from which their correct storage bucket can be determined through use of the modulo operator, which returns the remainder of a division operation (if i have 16 memory “buckets” and I divide the unique number for this object and get a remainder of 5, I can store it, and look for it later, in the fifth bucket, which will make accessing times constant as we will only have to look through the elements stored in that bucket, and since we increase our number of buckets proportional to the number of objects we store, our average list length will be ~1 item).  The LRU cache is then a combination of this hash map and another linked list, where the linked list represents the items stored in memory, and each time we access an item we move it to the front of the linked list.  Once we have too many items to keep in memory, this list will tell us which items we have used least recently so that we can remove them from our RAM.  The hash is necessary to keep our lookup times down, as we can have the return value for various lookup keys be the actual objects stored in the LRU cache linked list (see above for my brief explanation of why looking things up in a hash is faster). 

Hash Map

Implementation:

contains buckets

hash key to determine which bucket to place key

allows O(1) look up for best case since hash function knows exactly which one bucket to look in

hash function == mod by count of buckets

rehashing

amortization: double number of buckets when limit reached

hash to rehash to figure out buckets again since bucket count has changed => hash function different

worst case O(n), but average O(1) 

More to learn:

how are Javascript objects differently built?

mumur hash in ruby

Linked List

a series of nodes that have pointers to another node

Sentinel nodes: 

specifically designated node used as a traversal path terminator

commonly used with linked lists and trees

used over using `nil` object

allows less type comparison, ie avoid following:

               if node.nil?

as result, advantage: 

increased speed of operations

reduced algorithmic complexity and code size

Doubly linked list useful in caching

LRU (Least Recently Used) Cache

what is it: algorithm that stores n capcity of items and ejects the least recently used item when capacity reached

Key point:

what to cache =  time_to_make * frequency_visited

Premise:

check the cache and move to top

go get it and insert

eject the LRU

Implementation:

hash map => O(1) look-up but O(n) eject

linked list => O(1) eject but O(n) look-up

need to be doubly linked since add at head and eject at tail

The magic behind hash maps' O(1) insertion, deletion, and search times was finally dispelled today. By using an array as the underlying structure to store buckets of doubly-linked lists, I learned how to build a fully funcitonal hash map. First, I wrote a hash function using the XOR bitwise operator to generate a function with a good, deterministic distribution of keys for strings, arrays, and hashes themselves. Then, I wrote a Hash class that starts out with a small array of buckets that doubles in size each time the number of elements in the hash exceeds the number of buckets. Each time a new key-value pair is inserted into the hash, the key given by the aforementioned hash function is modded by the length of the underlying array, providing the hash map with an index into which to push a link storing the new key-value pair.

Buildling on this hash map, I also learned how to build a least recently used (LRU) cache. An LRU cache stores key-value pairs where the key is some costly operation and the value is the result of that operation. The idea is to store a small number of such pairs for operations that are frequently required; by storing the result of such a compuation in a hash once we've already computed it, we can then simply look up the value associated with the key provided by the operation itself when we need the result again, thereby providing us with the result of the operation in O(1) time without having to actually compute it again, saving our program a great deal of time.

However, since the number of operations a program or CPU computes is huge, we can only store a limited number of such key-value pairs in our LRU cache. Thus, when the number of such pairs in our cache exceeds the cache's maximum size, we simply throw-out the least recently-used pair (hence the name "least recently used cache"). To achieve a structure with all the aforementioned properties, I stored the key-value pairs in a doubly-linked list and stored references to each of the links within a hash map. This way, we can achieve O(1) search time for our linked list, since we can simply look up, in our hash, the reference to the link we want in O(1) time rather than having to iterate through each link in the list itself. When a new operation is computed, a link storing its key-value pair is put on the front of the list and a mapping of its key to that link is added to the hash map. If an operation already in the linked list is used again, we move that link to the front of the list. And if the list exceeds its maximum length, we simply remove the last (and thus least recently-used) link from the list and remove its key-value pair from the hash map.

Yesterday I did an implementation of an LRU Cache using a list and an unordered_map in C++. It's still buggy, but I wanted to save the solution in case I wanted to come back and debug it later. My best guess is that the segfault occurs in the usage of splice and maybe you can't splice to the end of a list. 

#include <list> #include <unordered_map>

using namespace std;

struct Node { int key, value; Node(int k, int v) : key(k), value(v) {}; };