org.mapdb

Class HTreeMap<K,V>

java.lang.Object

java.util.AbstractMap<K,V>

org.mapdb.HTreeMap<K,V>

All Implemented Interfaces:

java.util.concurrent.ConcurrentMap<K,V>, java.util.Map<K,V>, Bind.MapWithModificationListener<K,V>

public class HTreeMap<K,V>
extends java.util.AbstractMap<K,V>
implements java.util.concurrent.ConcurrentMap<K,V>, Bind.MapWithModificationListener<K,V>

Thread safe concurrent HashMap

This map uses full 32bit hash from beginning, There is no initial load factor and rehash. Technically it is not hash table, but hash tree with nodes expanding when they become full.

This map is suitable for number of records 1e9 and over. Larger number of records will increase hash collisions and performance will degrade linearly with number of records (separate chaining).

Concurrent scalability is achieved by splitting HashMap into 16 segments, each with separate lock. Very similar to ConcurrentHashMap

Author:

Jan Kotek

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) class	HTreeMap.Entry2
(package private) class	HTreeMap.EntryIterator
protected static class	HTreeMap.ExpireLinkNode
protected static class	HTreeMap.ExpireRunnable
(package private) class	HTreeMap.HashIterator
(package private) class	HTreeMap.KeyIterator
class	HTreeMap.KeySet
protected static class	HTreeMap.LinkedNode<K,V> node which holds key-value pair
(package private) class	HTreeMap.ValueIterator

Nested classes/interfaces inherited from class java.util.AbstractMap

java.util.AbstractMap.SimpleEntry<K,V>, java.util.AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

java.util.Map.Entry<K,V>

Field Summary

Fields

Modifier and Type	Field and Description
private java.util.Set<java.util.Map.Entry<K,V>>	_entrySet
private java.util.Set<K>	_keySet
private java.util.Collection<V>	_values
protected static int	BUCKET_OVERFLOW
protected java.util.concurrent.CountDownLatch	closeLatch
protected java.lang.Runnable	closeListener
protected Atomic.Long	counter
protected static Serializer<long[][]>	DIR_SERIALIZER
protected static int	DIV8
protected Engine	engine
protected long	expire
protected long	expireAccess
protected boolean	expireAccessFlag
protected boolean	expireFlag
protected long[]	expireHeads
protected long	expireMaxSize
protected boolean	expireMaxSizeFlag
protected long	expireStoreSize
protected long[]	expireTails
protected long	expireTimeStart
protected Hasher<K>	hasher
protected int	hashSalt Salt added to hash before rehashing, so it is harder to trigger hash collision attack.
protected boolean	hasValues is this a Map or Set? if false, entries do not have values, only keys are allowed
protected Serializer<K>	keySerializer
protected Serializer<HTreeMap.LinkedNode<K,V>>	LN_SERIALIZER
protected static int	MOD8
protected Bind.MapListener<K,V>[]	modListeners
protected java.lang.Object	modListenersLock
protected java.util.concurrent.locks.ReentrantReadWriteLock[]	segmentLocks
protected long[]	segmentRecids list of segments, this is immutable
protected Fun.Function1<V,K>	valueCreator
protected Serializer<V>	valueSerializer

Constructor Summary

Constructors

Constructor and Description
HTreeMap(Engine engine, long counterRecid, int hashSalt, long[] segmentRecids, Serializer<K> keySerializer, Serializer<V> valueSerializer, long expireTimeStart, long expire, long expireAccess, long expireMaxSize, long expireStoreSize, long[] expireHeads, long[] expireTails, Fun.Function1<V,K> valueCreator, Hasher hasher, boolean disableLocks) Opens HTreeMap

Method Summary

Modifier and Type	Method and Description
void	clear()
void	close() Closes underlying storage and releases all resources.
boolean	containsKey(java.lang.Object o)
boolean	containsValue(java.lang.Object value)
java.util.Set<java.util.Map.Entry<K,V>>	entrySet()
protected void	expireCheckSegment(int segment)
protected void	expireLinkAdd(int segment, long expireNodeRecid, long keyRecid, int hash)
protected void	expireLinkBump(int segment, long nodeRecid, boolean access)
protected HTreeMap.ExpireLinkNode	expireLinkRemove(int segment, long nodeRecid)
protected HTreeMap.ExpireLinkNode	expireLinkRemoveLast(int segment)
protected void	expirePurge()
protected void	expirePurgeSegment(int seg, long removePerSegment)
V	get(java.lang.Object o)
Engine	getEngine()
protected HTreeMap.LinkedNode<K,V>	getInner(java.lang.Object o, int h, int segment)
long	getMaxExpireTime() Returns maximal (newest) expiration timestamp
long	getMinExpireTime() Returns minimal (oldest) expiration timestamp
V	getPeek(java.lang.Object key) Return given value, without updating cache statistics if `expireAccess()` is true It also does not use `valueCreator` if value is not found (always returns null if not found)
protected int	hash(java.lang.Object key)
boolean	isEmpty()
java.util.Set<K>	keySet()
void	modificationListenerAdd(Bind.MapListener<K,V> listener) Add new modification listener notified when Map has been updated
void	modificationListenerRemove(Bind.MapListener<K,V> listener) Remove registered notification listener
protected void	notify(K key, V oldValue, V newValue)
protected static long[]	preallocateSegments(Engine engine)
V	put(K key, V value)
V	putIfAbsent(K key, V value)
private V	putInner(K key, V value, int h, int segment)
private void	recursiveDirClear(long dirRecid)
private long	recursiveDirCount(long dirRecid)
private void	recursiveDirDelete(int h, int level, long[] dirRecids, long[][] dir, int slot)
V	remove(java.lang.Object key)
boolean	remove(java.lang.Object key, java.lang.Object value)
protected V	removeInternal(java.lang.Object key, int segment, int h, boolean removeExpire)
V	replace(K key, V value)
boolean	replace(K key, V oldValue, V newValue)
int	size()
long	sizeLong()
java.util.Map<K,V>	snapshot() Make readonly snapshot view of current Map.
java.util.Collection<V>	values()

Methods inherited from class java.util.AbstractMap

clone, equals, hashCode, putAll, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.concurrent.ConcurrentMap

compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, replaceAll

Methods inherited from interface java.util.Map

equals, hashCode, putAll

Field Detail

BUCKET_OVERFLOW

protected static final int BUCKET_OVERFLOW

See Also:

Constant Field Values

DIV8

protected static final int DIV8

See Also:

Constant Field Values

MOD8

protected static final int MOD8

See Also:

Constant Field Values

hasValues

protected final boolean hasValues

is this a Map or Set? if false, entries do not have values, only keys are allowed

hashSalt

protected final int hashSalt

Salt added to hash before rehashing, so it is harder to trigger hash collision attack.

counter

protected final Atomic.Long counter

keySerializer

protected final Serializer<K> keySerializer

valueSerializer

protected final Serializer<V> valueSerializer

hasher

protected final Hasher<K> hasher

engine

protected final Engine engine

expireFlag

protected final boolean expireFlag

expireTimeStart

protected final long expireTimeStart

expire

protected final long expire

expireAccessFlag

protected final boolean expireAccessFlag

expireAccess

protected final long expireAccess

expireMaxSize

protected final long expireMaxSize

expireStoreSize

protected final long expireStoreSize

expireMaxSizeFlag

protected final boolean expireMaxSizeFlag

expireHeads

protected final long[] expireHeads

expireTails

protected final long[] expireTails

valueCreator

protected final Fun.Function1<V,K> valueCreator

closeLatch

protected final java.util.concurrent.CountDownLatch closeLatch

closeListener

protected final java.lang.Runnable closeListener

LN_SERIALIZER

protected final Serializer<HTreeMap.LinkedNode<K,V>> LN_SERIALIZER

DIR_SERIALIZER

protected static final Serializer<long[][]> DIR_SERIALIZER

segmentRecids

protected final long[] segmentRecids

list of segments, this is immutable

segmentLocks

protected final java.util.concurrent.locks.ReentrantReadWriteLock[] segmentLocks

_keySet

private final java.util.Set<K> _keySet

_values

private final java.util.Collection<V> _values

_entrySet

private final java.util.Set<java.util.Map.Entry<K,V>> _entrySet

modListenersLock

protected final java.lang.Object modListenersLock

modListeners

protected Bind.MapListener<K,V>[] modListeners

Constructor Detail

HTreeMap

public HTreeMap(Engine engine,
                long counterRecid,
                int hashSalt,
                long[] segmentRecids,
                Serializer<K> keySerializer,
                Serializer<V> valueSerializer,
                long expireTimeStart,
                long expire,
                long expireAccess,
                long expireMaxSize,
                long expireStoreSize,
                long[] expireHeads,
                long[] expireTails,
                Fun.Function1<V,K> valueCreator,
                Hasher hasher,
                boolean disableLocks)

Opens HTreeMap

Method Detail

preallocateSegments

protected static long[] preallocateSegments(Engine engine)

containsKey

public boolean containsKey(java.lang.Object o)

Specified by:

containsKey in interface java.util.Map<K,V>

Overrides:

containsKey in class java.util.AbstractMap<K,V>

size

public int size()

Specified by:

size in interface java.util.Map<K,V>

Overrides:

size in class java.util.AbstractMap<K,V>

sizeLong

public long sizeLong()

Specified by:

sizeLong in interface Bind.MapWithModificationListener<K,V>

Returns:

size of map, but in 64bit long which does not overflow at 2e9 items.

recursiveDirCount

private long recursiveDirCount(long dirRecid)

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface java.util.Map<K,V>

Overrides:

isEmpty in class java.util.AbstractMap<K,V>

get

public V get(java.lang.Object o)

Specified by:

get in interface java.util.Map<K,V>

Overrides:

get in class java.util.AbstractMap<K,V>

getPeek

public V getPeek(java.lang.Object key)

Return given value, without updating cache statistics if `expireAccess()` is true It also does not use `valueCreator` if value is not found (always returns null if not found)

Parameters:

key - key to lookup

Returns:

value associated with key or null

getInner

protected HTreeMap.LinkedNode<K,V> getInner(java.lang.Object o,
                                            int h,
                                            int segment)

put

public V put(K key,
             V value)

Specified by:

put in interface java.util.Map<K,V>

Overrides:

put in class java.util.AbstractMap<K,V>

putInner

private V putInner(K key,
                   V value,
                   int h,
                   int segment)

remove

public V remove(java.lang.Object key)

Specified by:

remove in interface java.util.Map<K,V>

Overrides:

remove in class java.util.AbstractMap<K,V>

removeInternal

protected V removeInternal(java.lang.Object key,
                           int segment,
                           int h,
                           boolean removeExpire)

recursiveDirDelete

private void recursiveDirDelete(int h,
                                int level,
                                long[] dirRecids,
                                long[][] dir,
                                int slot)

clear

public void clear()

Specified by:

clear in interface java.util.Map<K,V>

Overrides:

clear in class java.util.AbstractMap<K,V>

recursiveDirClear

private void recursiveDirClear(long dirRecid)

containsValue

public boolean containsValue(java.lang.Object value)

Specified by:

containsValue in interface java.util.Map<K,V>

Overrides:

containsValue in class java.util.AbstractMap<K,V>

keySet

public java.util.Set<K> keySet()

Specified by:

keySet in interface java.util.Map<K,V>

Overrides:

keySet in class java.util.AbstractMap<K,V>

values

public java.util.Collection<V> values()

Specified by:

values in interface java.util.Map<K,V>

Overrides:

values in class java.util.AbstractMap<K,V>

entrySet

public java.util.Set<java.util.Map.Entry<K,V>> entrySet()

Specified by:

entrySet in interface java.util.Map<K,V>

Specified by:

entrySet in class java.util.AbstractMap<K,V>

hash

protected int hash(java.lang.Object key)

putIfAbsent

public V putIfAbsent(K key,
                     V value)

Specified by:

putIfAbsent in interface java.util.concurrent.ConcurrentMap<K,V>

Specified by:

putIfAbsent in interface java.util.Map<K,V>

remove

public boolean remove(java.lang.Object key,
                      java.lang.Object value)

Specified by:

remove in interface java.util.concurrent.ConcurrentMap<K,V>

Specified by:

remove in interface java.util.Map<K,V>

replace

public boolean replace(K key,
                       V oldValue,
                       V newValue)

Specified by:

replace in interface java.util.concurrent.ConcurrentMap<K,V>

Specified by:

replace in interface java.util.Map<K,V>

replace

public V replace(K key,
                 V value)

Specified by:

replace in interface java.util.concurrent.ConcurrentMap<K,V>

Specified by:

replace in interface java.util.Map<K,V>

expireLinkAdd

protected void expireLinkAdd(int segment,
                             long expireNodeRecid,
                             long keyRecid,
                             int hash)

expireLinkBump

protected void expireLinkBump(int segment,
                              long nodeRecid,
                              boolean access)

expireLinkRemoveLast

protected HTreeMap.ExpireLinkNode expireLinkRemoveLast(int segment)

expireLinkRemove

protected HTreeMap.ExpireLinkNode expireLinkRemove(int segment,
                                                   long nodeRecid)

getMaxExpireTime

public long getMaxExpireTime()

Returns maximal (newest) expiration timestamp

getMinExpireTime

public long getMinExpireTime()

Returns minimal (oldest) expiration timestamp

expirePurge

protected void expirePurge()

expirePurgeSegment

protected void expirePurgeSegment(int seg,
                                  long removePerSegment)

expireCheckSegment

protected void expireCheckSegment(int segment)

snapshot

public java.util.Map<K,V> snapshot()

Make readonly snapshot view of current Map. Snapshot is immutable and not affected by modifications made by other threads. Useful if you need consistent view on Map.

Maintaining snapshot have some overhead, underlying Engine is closed after Map view is GCed. Please make sure to release reference to this Map view, so snapshot view can be garbage collected.

Returns:

snapshot

modificationListenerAdd

public void modificationListenerAdd(Bind.MapListener<K,V> listener)

Description copied from interface: Bind.MapWithModificationListener

Add new modification listener notified when Map has been updated

Specified by:

modificationListenerAdd in interface Bind.MapWithModificationListener<K,V>

Parameters:

listener - callback interface notified when map changes

modificationListenerRemove

public void modificationListenerRemove(Bind.MapListener<K,V> listener)

Description copied from interface: Bind.MapWithModificationListener

Remove registered notification listener

Specified by:

modificationListenerRemove in interface Bind.MapWithModificationListener<K,V>

Parameters:

listener - callback interface notified when map changes

notify

protected void notify(K key,
                      V oldValue,
                      V newValue)

close

public void close()

Closes underlying storage and releases all resources. Used mostly with temporary collections where engine is not accessible.

getEngine

public Engine getEngine()