com.sleepycat.je.cleaner

Class UtilizationProfile

java.lang.Object

com.sleepycat.je.cleaner.UtilizationProfile

All Implemented Interfaces:

EnvConfigObserver

public class UtilizationProfile
extends java.lang.Object
implements EnvConfigObserver

The UP tracks utilization summary information for all log files.

Unlike the UtilizationTracker, the UP is not accessed under the log write latch and is instead synchronized on itself for protecting the cache. It is not accessed during the primary data access path, except for when flushing (writing) file summary LNs. This occurs in the following cases:

1. The summary information is flushed at the end of a checkpoint. This allows tracking to occur in memory in between checkpoints, and replayed during recovery.
2. When committing the truncateDatabase and removeDatabase operations, the summary information is flushed because detail tracking for those operations is not replayed during recovery
3. The evictor will ask the UtilizationTracker to flush the largest summary if the memory taken by the tracker exeeds its budget.

The cache is populated by the RecoveryManager just before performing the initial checkpoint. The UP must be open and populated in order to respond to requests to flush summaries and to evict tracked detail, even if the cleaner is disabled.

WARNING: While synchronized on this object, eviction is not permitted. If it were, this could cause deadlocks because the order of locking would be the UP object and then the evictor. During normal eviction the order is to first lock the evictor and then the UP, when evicting tracked detail.

The methods in this class synchronize to protect the cached summary information. Some methods also access the UP database. However, because eviction must not occur while synchronized, UP database access is not performed while synchronized except in one case: when inserting a new summary record. In that case we disallow eviction during the database operation.

Field Summary

Fields

Modifier and Type	Field and Description
(package private) int	minAge Minumum age to qualify for cleaning.
(package private) int	minFileUtilization Minimum utilization threshold for an individual log file that triggers cleaning.
(package private) int	minUtilization Minimum overall utilization threshold that triggers cleaning.

Constructor Summary

Constructors

Constructor and Description
UtilizationProfile(EnvironmentImpl env, UtilizationTracker tracker) Creates an empty UP.

Method Summary

Methods

Modifier and Type	Method and Description
void	clearCache() Clears the cache of file summary info.
(package private) void	close() Update memory budgets when this profile is closed and will never be accessed again.
void	envConfigUpdate(DbConfigManager cm, EnvironmentMutableConfig ignore) Process notifications of mutable property changes.
void	flushFileSummary(TrackedFileSummary tfs) Updates and stores the FileSummary for a given tracked file, if flushing of the summary is allowed.
void	flushFileUtilization(java.util.Collection<TrackedFileSummary> activeFiles) Flush a FileSummaryLN node for each TrackedFileSummary that is currently active in the given tracker.
void	flushLocalTracker(LocalUtilizationTracker localTracker) Count the given locally tracked info as obsolete and then log the file and database info..
(package private) java.lang.Long	getBestFileForCleaning(FileSelector fileSelector, boolean forceCleaning, java.util.Set<java.lang.Long> lowUtilizationFiles, boolean isBacklog) Returns the best file that qualifies for cleaning, or null if no file qualifies.
(package private) java.lang.Long	getCheapestFileToClean(java.util.SortedSet<java.lang.Long> files) Returns the cheapest file to clean from the given list of files.
DatabaseImpl	getFileSummaryDb() For unit testing.
java.util.SortedMap<java.lang.Long,FileSummary>	getFileSummaryMap(boolean includeTrackedFiles) Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests.
(package private) int	getNumberOfFiles() Returns the number of files in the profile.
(package private) TrackedFileSummary	getObsoleteDetail(java.lang.Long fileNum, PackedOffsets packedOffsets, boolean logUpdate) Returns the stored/packed obsolete offsets and the tracked obsolete offsets for the given file.
(package private) long	getTotalLogSize() Returns an approximation of the total log size.
boolean	isRMWFixEnabled()
boolean	populateCache() Populate the profile for file selection.
(package private) void	removeFile(java.lang.Long fileNum, java.util.Set<DatabaseId> databases) Removes a file from the utilization database and the profile, after it has been deleted by the cleaner.
static int	utilization(long obsoleteSize, long totalSize) Calculate the utilization percentage.
boolean	verifyFileSummaryDatabase() Checks that all FSLN offsets are indeed obsolete.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

minUtilization

int minUtilization

Minimum overall utilization threshold that triggers cleaning. Is non-private for unit tests.

minFileUtilization

int minFileUtilization

Minimum utilization threshold for an individual log file that triggers cleaning. Is non-private for unit tests.

minAge

int minAge

Minumum age to qualify for cleaning. If the first active LSN file is 5 and the mininum age is 2, file 4 won't qualify but file 3 will. Must be greater than zero because we never clean the first active LSN file. Is non-private for unit tests.

Constructor Detail

UtilizationProfile

public UtilizationProfile(EnvironmentImpl env,
                  UtilizationTracker tracker)
                   throws DatabaseException

Creates an empty UP.

Throws:

DatabaseException

Method Detail

envConfigUpdate

public void envConfigUpdate(DbConfigManager cm,
                   EnvironmentMutableConfig ignore)
                     throws DatabaseException

Process notifications of mutable property changes.

Specified by:

envConfigUpdate in interface EnvConfigObserver

Throws:

DatabaseException

isRMWFixEnabled

public boolean isRMWFixEnabled()

See Also:

EnvironmentParams.CLEANER_RMW_FIX, FileSummaryLN.postFetchInit(com.sleepycat.je.dbi.DatabaseImpl, long)

getNumberOfFiles

int getNumberOfFiles()
               throws DatabaseException

Returns the number of files in the profile.

Throws:

DatabaseException

getTotalLogSize

long getTotalLogSize()

Returns an approximation of the total log size. Used for stats.

getCheapestFileToClean

java.lang.Long getCheapestFileToClean(java.util.SortedSet<java.lang.Long> files)
                                throws DatabaseException

Returns the cheapest file to clean from the given list of files. This method is used to select the first file to be cleaned in the batch of to-be-cleaned files.

Throws:

DatabaseException

getBestFileForCleaning

java.lang.Long getBestFileForCleaning(FileSelector fileSelector,
                                    boolean forceCleaning,
                                    java.util.Set<java.lang.Long> lowUtilizationFiles,
                                    boolean isBacklog)
                                throws DatabaseException

Returns the best file that qualifies for cleaning, or null if no file qualifies.

Parameters:

fileSelector - is used to determine valid cleaning candidates.

forceCleaning - is true to always select a file, even if its utilization is above the minimum utilization threshold.

lowUtilizationFiles - is a returned set of files that are below the minimum utilization threshold.

Throws:

DatabaseException

utilization

public static int utilization(long obsoleteSize,
              long totalSize)

Calculate the utilization percentage.

flushLocalTracker

public void flushLocalTracker(LocalUtilizationTracker localTracker)
                       throws DatabaseException

Count the given locally tracked info as obsolete and then log the file and database info..

Throws:

DatabaseException

flushFileUtilization

public void flushFileUtilization(java.util.Collection<TrackedFileSummary> activeFiles)
                          throws DatabaseException

Flush a FileSummaryLN node for each TrackedFileSummary that is currently active in the given tracker.

Throws:

DatabaseException

getFileSummaryMap

public java.util.SortedMap<java.lang.Long,FileSummary> getFileSummaryMap(boolean includeTrackedFiles)
                                                                  throws DatabaseException

Returns a copy of the current file summary map, optionally including tracked summary information, for use by the DbSpace utility and by unit tests. The returned map's key is a Long file number and its value is a FileSummary.

Throws:

DatabaseException

clearCache

public void clearCache()

Clears the cache of file summary info. The cache starts out unpopulated and is populated on the first call to getBestFileForCleaning.

removeFile

void removeFile(java.lang.Long fileNum,
              java.util.Set<DatabaseId> databases)
          throws DatabaseException

Removes a file from the utilization database and the profile, after it has been deleted by the cleaner.

Throws:

DatabaseException

flushFileSummary

public void flushFileSummary(TrackedFileSummary tfs)
                      throws DatabaseException

Updates and stores the FileSummary for a given tracked file, if flushing of the summary is allowed.

Throws:

DatabaseException

getObsoleteDetail

TrackedFileSummary getObsoleteDetail(java.lang.Long fileNum,
                                   PackedOffsets packedOffsets,
                                   boolean logUpdate)
                               throws DatabaseException

Returns the stored/packed obsolete offsets and the tracked obsolete offsets for the given file. The tracked summary object returned can be used to test for obsolete offsets that are being added during cleaning by other threads participating in lazy migration. The caller must call TrackedFileSummary.setAllowFlush(true) when cleaning is complete. This method performs eviction and is not synchronized.

Parameters:

logUpdate - if true, log any updates to the utilization profile. If false, only retrieve the new information.

Throws:

DatabaseException

populateCache

public boolean populateCache()
                      throws DatabaseException

Populate the profile for file selection. This method performs eviction and is not synchronized. It must be called before recovery is complete so that synchronization is unnecessary. It must be called before the recovery checkpoint so that the checkpoint can flush file summary information.

Throws:

DatabaseException

getFileSummaryDb

public DatabaseImpl getFileSummaryDb()

For unit testing.

verifyFileSummaryDatabase

public boolean verifyFileSummaryDatabase()
                                  throws DatabaseException

Checks that all FSLN offsets are indeed obsolete. Assumes that the system is quiesent (does not lock LNs). This method is not synchronized (because it doesn't access fileSummaryMap) and eviction is allowed.

Returns:

true if no verification failures.

Throws:

DatabaseException

close

void close()

Update memory budgets when this profile is closed and will never be accessed again.