org.apache.jackrabbit.core.data
Class CachingDataStore

java.lang.Object
  org.apache.jackrabbit.core.data.AbstractDataStore
      org.apache.jackrabbit.core.data.CachingDataStore

All Implemented Interfaces:

AsyncUploadCallback, DataStore, MultiDataStoreAware

public abstract class CachingDataStore
extends AbstractDataStore
implements MultiDataStoreAware, AsyncUploadCallback

A caching data store that consists of LocalCache and Backend. Backend is single source of truth. All methods first try to fetch information from LocalCache. If record is not available in LocalCache, then it is fetched from Backend and saved to LocalCache for further access. This class is designed to work without LocalCache and then all information is fetched from Backend. To disable LocalCache set setCacheSize(long) to 0. * Configuration:

 <DataStore class="org.apache.jackrabbit.aws.ext.ds.CachingDataStore">
 
     <param name="path" value="/data/datastore"/>
     <param name="config" value="${rep.home}/backend.properties"/>
     <param name="cacheSize" value="68719476736"/>
     <param name="secret" value="123456"/>
     <param name="setCachePurgeTrigFactor(double)" value="0.95d"/>
     <param name="cacheSize" value="0.85d"/>
     <param name="minRecordLength" value="1024"/>
     <param name="continueOnAsyncUploadFailure" value="false"/>
     <param name="concurrentUploadsThreads" value="10"/>
     <param name="asyncUploadLimit" value="100"/>
     <param name="uploadRetries" value="3"/>
 </DataStore>









Field Summary




protected  Backend
backend



           



protected  Map<DataIdentifier,WeakReference<DataIdentifier>>
inUse



          All data identifiers that are currently in use are in this set until they
 are garbage collected.



protected  Map<DataIdentifier,Integer>
uploadRetryMap



          In memory map to hold failed asynchronous upload DataIdentifier
 and its retry count.


 



Constructor Summary
CachingDataStore()



           
 



Method Summary




 DataRecord
addRecord(InputStream input)



          Creates a new data record in Backend.



 void
clearInUse()



          Clear the in-use list.



 void
close()



          Close the data store



protected abstract  Backend
createBackend()



           



 int
deleteAllOlderThan(long min)



          Delete objects that have a modified date older than the specified date.



 void
deleteRecord(DataIdentifier identifier)



          This method deletes record from Backend and then from
 LocalCache



 Iterator<DataIdentifier>
getAllIdentifiers()



          Retrieves all identifiers from Backend.



 int
getAsyncUploadLimit()



           



 Backend
getBackend()



           



 double
getCachePurgeResizeFactor()



           



 double
getCachePurgeTrigFactor()



           



 long
getCacheSize()



           



 int
getConcurrentUploadsThreads()



           



 String
getConfig()



          Return path of configuration properties.



 long
getLastModified(DataIdentifier identifier)



          Return lastModified of record from Backend assuming
 Backend as a single source of truth.



 long
getLength(DataIdentifier identifier)



          Return the length of record from LocalCache if available,
 otherwise retrieve it from Backend.



protected abstract  String
getMarkerFile()



           



 int
getMinRecordLength()



          Return mininum object length.



protected  byte[]
getOrCreateReferenceKey()



          Returns the reference key of this data store.



 String
getPath()



           



 Set<String>
getPendingUploads()



           



 DataRecord
getRecord(DataIdentifier identifier)



          Returns the identified data record.



 DataRecord
getRecordIfStored(DataIdentifier identifier)



          Get a data record for the given identifier or null it data record doesn't
 exist in Backend



 int
getUploadRetries()



           



 void
init(String homeDir)



          Initialized the data store.



 boolean
isContinueOnAsyncUploadFailure()



           



 boolean
isInUse(DataIdentifier identifier)



           



 void
onAbort(AsyncUploadResult result)



          Callback method for aborted asynchronous upload.



 void
onFailure(AsyncUploadResult result)



          Callback method for failed asynchronous upload.



 void
onSuccess(AsyncUploadResult result)



          Callback method for successful asynchronous upload.



 void
setAsyncUploadLimit(int asyncUploadLimit)



           



 void
setCachePurgeResizeFactor(double cachePurgeResizeFactor)



          Set purge resize factor of LocalCache.



 void
setCachePurgeTrigFactor(double cachePurgeTrigFactor)



          Set purge trigger factor of LocalCache.



 void
setCacheSize(long cacheSize)



          Set size of LocalCache.



 void
setConcurrentUploadsThreads(int concurrentUploadsThreads)



           



 void
setConfig(String config)



          Set the configuration properties path.



 void
setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)



           



 void
setMinRecordLength(int minRecordLength)



          Set the minimum object length.



 void
setPath(String path)



          Set path of LocalCache.



 void
setSecret(String secret)



          Setter for configuration based secret



 void
setUploadRetries(int uploadRetries)



           



 void
updateModifiedDateOnAccess(long before)



          From now on, update the modified date of an object even when accessing it.


 
Methods inherited from class org.apache.jackrabbit.core.data.AbstractDataStore
encodeHexString, getRecordFromReference, getReferenceFromIdentifier
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 





Field Detail



inUse
protected Map<DataIdentifier,WeakReference<DataIdentifier>> inUse

All data identifiers that are currently in use are in this set until they
 are garbage collected.







uploadRetryMap
protected final Map<DataIdentifier,Integer> uploadRetryMap

In memory map to hold failed asynchronous upload DataIdentifier
 and its retry count. Once if all retries are exhausted or file is 
 successfully uploaded, then corresponding entry is flushed from the map.
 As all failed uploads are synchronously uploaded at startup, this map 
 is not required to be persisted.







backend
protected Backend backend








Constructor Detail



CachingDataStore
public CachingDataStore()






Method Detail



createBackend
protected abstract Backend createBackend()











getMarkerFile
protected abstract String getMarkerFile()











init
public void init(String homeDir)
          throws RepositoryException

Initialized the data store. If the path is not set, <repository
 home>/repository/datastore is used. This directory is automatically
 created if it does not yet exist. During first initialization, it upload
 all files from local datastore to backed and local datastore act as a
 local cache.


Specified by:
init in interface DataStore


Parameters:
homeDir - the home directory of the repository
Throws:
RepositoryException





addRecord
public DataRecord addRecord(InputStream input)
                     throws DataStoreException

Creates a new data record in Backend. The stream is first
 consumed and the contents are saved in a temporary file and the SHA-1
 message digest of the stream is calculated. If a record with the same
 SHA-1 digest (and length) is found then it is returned. Otherwise new
 record is created in Backend and the temporary file is moved in
 place to LocalCache.


Specified by:
addRecord in interface DataStore


Parameters:
input - binary stream
Returns:
CachingDataRecord
Throws:
DataStoreException - if the record could not be created.





getRecord
public DataRecord getRecord(DataIdentifier identifier)
                     throws DataStoreException

Description copied from interface: DataStore
Returns the identified data record. The given identifier should be
 the identifier of a previously saved data record. Since records are
 never removed, there should never be cases where the identified record
 is not found. Abnormal cases like that are treated as errors and
 handled by throwing an exception.


Specified by:
getRecord in interface DataStore
Overrides:
getRecord in class AbstractDataStore


Parameters:
identifier - data identifier
Returns:
identified data record
Throws:
DataStoreException - if the data store could not be accessed,
                     or if the given identifier is invalid





getRecordIfStored
public DataRecord getRecordIfStored(DataIdentifier identifier)
                             throws DataStoreException

Get a data record for the given identifier or null it data record doesn't
 exist in Backend


Specified by:
getRecordIfStored in interface DataStore


Parameters:
identifier - identifier of record.
Returns:
the CachingDataRecord or null.
Throws:
DataStoreException - if the data store could not be accessed





updateModifiedDateOnAccess
public void updateModifiedDateOnAccess(long before)

Description copied from interface: DataStore
From now on, update the modified date of an object even when accessing it.
 Usually, the modified date is only updated when creating a new object,
 or when a new link is added to an existing object. When this setting is enabled,
 even getLength() will update the modified date.


Specified by:
updateModifiedDateOnAccess in interface DataStore


Parameters:
before - - update the modified date to the current time if it is older than this value





getAllIdentifiers
public Iterator<DataIdentifier> getAllIdentifiers()
                                           throws DataStoreException

Retrieves all identifiers from Backend.


Specified by:
getAllIdentifiers in interface DataStore



Returns:
an iterator over all DataIdentifier objects
Throws:
DataStoreException - if the list could not be read





deleteRecord
public void deleteRecord(DataIdentifier identifier)
                  throws DataStoreException

This method deletes record from Backend and then from
 LocalCache


Specified by:
deleteRecord in interface MultiDataStoreAware


Parameters:
identifier - data identifier
Throws:
DataStoreException - if the data store could not be accessed, or if the given
             identifier is invalid





deleteAllOlderThan
public int deleteAllOlderThan(long min)
                       throws DataStoreException

Description copied from interface: DataStore
Delete objects that have a modified date older than the specified date.


Specified by:
deleteAllOlderThan in interface DataStore


Parameters:
min - the minimum time
Returns:
the number of data records deleted
Throws:
DataStoreException





getLastModified
public long getLastModified(DataIdentifier identifier)
                     throws DataStoreException

Return lastModified of record from Backend assuming
 Backend as a single source of truth.






Throws:
DataStoreException





getLength
public long getLength(DataIdentifier identifier)
               throws DataStoreException

Return the length of record from LocalCache if available,
 otherwise retrieve it from Backend.






Throws:
DataStoreException





getOrCreateReferenceKey
protected byte[] getOrCreateReferenceKey()
                                  throws DataStoreException

Description copied from class: AbstractDataStore
Returns the reference key of this data store. If one does not already
 exist, it is automatically created in an implementation-specific way.
 The default implementation simply creates a temporary random key that's
 valid only until the data store gets restarted. Subclasses can override
 and/or decorate this method to support a more persistent reference key.
 

 This method is called only once during the lifetime of a data store
 instance and the return value is cached in memory, so it's no problem
 if the implementation is slow.


Overrides:
getOrCreateReferenceKey in class AbstractDataStore



Returns:
reference key
Throws:
DataStoreException - if the key is not available





getPendingUploads
public Set<String> getPendingUploads()











onSuccess
public void onSuccess(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback
Callback method for successful asynchronous upload.


Specified by:
onSuccess in interface AsyncUploadCallback








onFailure
public void onFailure(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback
Callback method for failed asynchronous upload.


Specified by:
onFailure in interface AsyncUploadCallback








onAbort
public void onAbort(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback
Callback method for aborted asynchronous upload.


Specified by:
onAbort in interface AsyncUploadCallback








clearInUse
public void clearInUse()

Description copied from interface: DataStore
Clear the in-use list. This is only used for testing to make the the garbage collection
 think that objects are no longer in use.


Specified by:
clearInUse in interface DataStore








isInUse
public boolean isInUse(DataIdentifier identifier)











close
public void close()
           throws DataStoreException

Description copied from interface: DataStore
Close the data store


Specified by:
close in interface DataStore



Throws:
DataStoreException - if a problem occurred





setSecret
public void setSecret(String secret)

Setter for configuration based secret





Parameters:
secret - the secret used to sign reference binaries





setMinRecordLength
public void setMinRecordLength(int minRecordLength)

Set the minimum object length.





Parameters:
minRecordLength - the length





getMinRecordLength
public int getMinRecordLength()

Return mininum object length.


Specified by:
getMinRecordLength in interface DataStore



Returns:
the minimum size in bytes





getConfig
public String getConfig()

Return path of configuration properties.






Returns:
path of configuration properties.





setConfig
public void setConfig(String config)

Set the configuration properties path.





Parameters:
config - path of configuration properties.





getCacheSize
public long getCacheSize()






Returns:
size of LocalCache.





setCacheSize
public void setCacheSize(long cacheSize)

Set size of LocalCache.





Parameters:
cacheSize - size of LocalCache.





getPath
public String getPath()






Returns:
path of LocalCache.





setPath
public void setPath(String path)

Set path of LocalCache.





Parameters:
path - of LocalCache.





getCachePurgeTrigFactor
public double getCachePurgeTrigFactor()






Returns:
Purge trigger factor of LocalCache.





setCachePurgeTrigFactor
public void setCachePurgeTrigFactor(double cachePurgeTrigFactor)

Set purge trigger factor of LocalCache.





Parameters:
cachePurgeTrigFactor - purge trigger factor.





getCachePurgeResizeFactor
public double getCachePurgeResizeFactor()






Returns:
Purge resize factor of LocalCache.





setCachePurgeResizeFactor
public void setCachePurgeResizeFactor(double cachePurgeResizeFactor)

Set purge resize factor of LocalCache.





Parameters:
cachePurgeResizeFactor - purge resize factor.





getConcurrentUploadsThreads
public int getConcurrentUploadsThreads()











setConcurrentUploadsThreads
public void setConcurrentUploadsThreads(int concurrentUploadsThreads)











getAsyncUploadLimit
public int getAsyncUploadLimit()











setAsyncUploadLimit
public void setAsyncUploadLimit(int asyncUploadLimit)











isContinueOnAsyncUploadFailure
public boolean isContinueOnAsyncUploadFailure()











setContinueOnAsyncUploadFailure
public void setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)











getUploadRetries
public int getUploadRetries()











setUploadRetries
public void setUploadRetries(int uploadRetries)











getBackend
public Backend getBackend()