com.sleepycat.je.log

Class FileManager

java.lang.Object

com.sleepycat.je.log.FileManager

public class FileManager
extends java.lang.Object

The FileManager presents the abstraction of one contiguous file. It doles out LSNs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileManager.FileMode
(package private) class	FileManager.LogEndFileDescriptor The LogEndFileDescriptor is used to write and fsync the end of the log.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BAD_SUFFIX
static java.lang.String	DEL_SUFFIX
(package private) static java.lang.String[]	DEL_SUFFIXES
(package private) static boolean	IO_EXCEPTION_TESTING_ON_READ
(package private) static boolean	IO_EXCEPTION_TESTING_ON_WRITE
static java.lang.String	JE_SUFFIX
(package private) static java.lang.String[]	JE_SUFFIXES
(package private) long	lastFileNumberTouched
(package private) long	lastFileTouchedOffset
static long	N_BAD_WRITES
(package private) int	nFileOpens
(package private) long	nRandomReadBytes
(package private) long	nRandomReads
(package private) long	nRandomWriteBytes
(package private) long	nRandomWrites
(package private) long	nSequentialReadBytes
(package private) long	nSequentialReads
(package private) long	nSequentialWriteBytes
(package private) long	nSequentialWrites
static boolean	RUNRECOVERY_EXCEPTION_TESTING
static long	STOP_ON_WRITE_COUNT
static boolean	THROW_ON_WRITE
(package private) static boolean	THROW_RRE_FOR_UNIT_TESTS
boolean	VERIFY_CHECKSUMS
static long	WRITE_COUNT

Constructor Summary

Constructors

Constructor and Description
FileManager(EnvironmentImpl envImpl, java.io.File dbEnvHome, boolean readOnly) Set up the file cache and initialize the file manager to point to the beginning of the log.

Method Summary

Methods

Modifier and Type	Method and Description
(package private) boolean	bumpLsn(long size) Increase the current log position by "size" bytes.
boolean	checkEnvHomePermissions(boolean readOnly) Ensure that if the environment home dir is on readonly media or in a readonly directory that the environment has been opened for readonly access.
void	clear() Close all file handles and empty the cache.
void	close() Clear the file lock.
void	deleteFile(long fileNum) Delete log file NNNNNNNN.
boolean	filesExist()
static int	firstLogEntryOffset()
(package private) void	forceNewLogFile() Set the flag that causes a new file to be written before the next write.
java.lang.Long[]	getAllFileNumbers() Get all JE file numbers.
(package private) java.util.Set<java.lang.Long>	getCacheKeys()
long	getCurrentFileNum() Returns the highest (current) file number.
(package private) FileHandle	getFileHandle(long fileNum) Return a read only file handle that corresponds the this file number.
(package private) long	getFileHeaderPrevOffset(long fileNum)
int	getFileLogVersion(long fileNum) Returns the log version for the given file.
static java.lang.String	getFileName(long fileNum, java.lang.String suffix)
java.lang.Long	getFirstFileNum() public for cleaner.
java.lang.Long	getFollowingFileNum(long currentFileNum, boolean forward) Get the next file number before/after currentFileNum.
java.lang.String	getFullFileName(long fileNum, java.lang.String suffix)
(package private) java.lang.String[]	getFullFileNames(long fileNum)
java.lang.Long	getLastFileNum()
long	getLastUsedLsn() Return the last allocated LSN in the log.
long	getNextLsn() Return the next available LSN in the log.
long	getNFSyncRequests()
long	getNFSyncs()
long	getNFSyncTimeouts()
java.lang.Long	getNumFromName(java.lang.String fileName) Get the file number from a file name.
(package private) long	getPrevEntryOffset()
boolean	getReadOnly()
(package private) void	groupSync() Flush a file using the group sync mechanism, trying to amortize off other syncs.
boolean	isFileValid(long fileNum) For assertions that check whether a file is valid or has been deleted via log cleaning.
static java.lang.String[]	listFiles(java.io.File envDirFile, java.lang.String[] suffixes) Find JE files, flavor for unit test support.
java.lang.String[]	listFiles(long minFileNumber, long maxFileNumber) Find .jdb files which are >= the minimimum file number and <= the maximum file number.
java.lang.String[]	listFiles(java.lang.String[] suffixes) Find JE files.
(package private) void	loadStats(StatsConfig config, EnvironmentStats stats)
boolean	lockEnvironment(boolean readOnly, boolean exclusive) Lock the environment.
(package private) void	readFromFile(java.io.RandomAccessFile file, java.nio.ByteBuffer readBuffer, long offset, long fileNo) Read a buffer from a file at a given offset, using NIO if so configured.
void	releaseExclusiveLock()
static void	removeFiles(java.io.File envFile) Remove files from the environment home directory.
void	renameFile(long fileNum, java.lang.String newSuffix) Rename this file to NNNNNNNN.suffix.
(package private) void	restoreLastPosition()
(package private) void	saveLastPosition()
void	setIncludeDeletedFiles(boolean includeDeletedFiles)
void	setLastPosition(long nextAvailableLsn, long lastUsedLsn, long prevOffset) Set the file manager's "end of log".
void	setSyncAtFileEnd(boolean sync) May be used to disable sync at file end to speed unit tests.
(package private) void	syncLogEnd() FSync the end of the log.
(package private) void	syncLogEndAndFinishFile() Sync the end of the log, close off this log file.
void	truncateLog(long fileNum, long offset) Truncate a log at this position.
(package private) void	writeLogBuffer(LogBuffer fullBuffer) Write out a log buffer to the file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

IO_EXCEPTION_TESTING_ON_WRITE

static boolean IO_EXCEPTION_TESTING_ON_WRITE

IO_EXCEPTION_TESTING_ON_READ

static boolean IO_EXCEPTION_TESTING_ON_READ

THROW_RRE_FOR_UNIT_TESTS

static boolean THROW_RRE_FOR_UNIT_TESTS

WRITE_COUNT

public static long WRITE_COUNT

STOP_ON_WRITE_COUNT

public static long STOP_ON_WRITE_COUNT

N_BAD_WRITES

public static long N_BAD_WRITES

THROW_ON_WRITE

public static boolean THROW_ON_WRITE

JE_SUFFIX

public static final java.lang.String JE_SUFFIX

See Also:

Constant Field Values

DEL_SUFFIX

public static final java.lang.String DEL_SUFFIX

See Also:

Constant Field Values

BAD_SUFFIX

public static final java.lang.String BAD_SUFFIX

See Also:

Constant Field Values

DEL_SUFFIXES

static final java.lang.String[] DEL_SUFFIXES

JE_SUFFIXES

static final java.lang.String[] JE_SUFFIXES

VERIFY_CHECKSUMS

public boolean VERIFY_CHECKSUMS

lastFileNumberTouched

long lastFileNumberTouched

lastFileTouchedOffset

long lastFileTouchedOffset

nRandomReads

long nRandomReads

nRandomWrites

long nRandomWrites

nSequentialReads

long nSequentialReads

nSequentialWrites

long nSequentialWrites

nRandomReadBytes

long nRandomReadBytes

nRandomWriteBytes

long nRandomWriteBytes

nSequentialReadBytes

long nSequentialReadBytes

nSequentialWriteBytes

long nSequentialWriteBytes

nFileOpens

int nFileOpens

RUNRECOVERY_EXCEPTION_TESTING

public static final boolean RUNRECOVERY_EXCEPTION_TESTING

Constructor Detail

FileManager

public FileManager(EnvironmentImpl envImpl,
           java.io.File dbEnvHome,
           boolean readOnly)
            throws DatabaseException

Set up the file cache and initialize the file manager to point to the beginning of the log.

Parameters:

configManager -

dbEnvHome - environment home directory

Throws:

DatabaseException

Method Detail

setLastPosition

public void setLastPosition(long nextAvailableLsn,
                   long lastUsedLsn,
                   long prevOffset)

Set the file manager's "end of log".

Parameters:

nextAvailableLsn - LSN to be used for the next log entry

lastUsedLsn - last LSN to have a valid entry, may be null

prevOffset - value to use for the prevOffset of the next entry. If the beginning of the file, this is 0.

saveLastPosition

void saveLastPosition()

restoreLastPosition

void restoreLastPosition()

setSyncAtFileEnd

public void setSyncAtFileEnd(boolean sync)

May be used to disable sync at file end to speed unit tests. Must only be used for unit testing, since log corruption may result.

getFirstFileNum

public java.lang.Long getFirstFileNum()

public for cleaner.

Returns:

the number of the first file in this environment.

getReadOnly

public boolean getReadOnly()

getLastFileNum

public java.lang.Long getLastFileNum()

Returns:

the number of the last file in this environment.

getCurrentFileNum

public long getCurrentFileNum()

Returns the highest (current) file number. Because a long value cannot be read atomically without synchronization, this method should be called while holding the log write latch.

isFileValid

public boolean isFileValid(long fileNum)

For assertions that check whether a file is valid or has been deleted via log cleaning.

setIncludeDeletedFiles

public void setIncludeDeletedFiles(boolean includeDeletedFiles)

getAllFileNumbers

public java.lang.Long[] getAllFileNumbers()

Get all JE file numbers.

Returns:

an array of all JE file numbers.

getFollowingFileNum

public java.lang.Long getFollowingFileNum(long currentFileNum,
                                 boolean forward)

Get the next file number before/after currentFileNum.

Parameters:

currentFileNum - the file we're at right now. Note that it may not exist, if it's been cleaned and renamed.

forward - if true, we want the next larger file, if false we want the previous file

Returns:

null if there is no following file, or if filenum doesn't exist

filesExist

public boolean filesExist()

Returns:

true if there are any files at all.

getNumFromName

public java.lang.Long getNumFromName(java.lang.String fileName)

Get the file number from a file name.

Parameters:

the - file name

Returns:

the file number

listFiles

public java.lang.String[] listFiles(java.lang.String[] suffixes)

Find JE files. Return names sorted in ascending fashion.

Parameters:

suffix - which type of file we're looking for

Returns:

array of file names

listFiles

public java.lang.String[] listFiles(long minFileNumber,
                           long maxFileNumber)

Find .jdb files which are >= the minimimum file number and <= the maximum file number. Return names sorted in ascending fashion.

Returns:

array of file names

listFiles

public static java.lang.String[] listFiles(java.io.File envDirFile,
                           java.lang.String[] suffixes)

Find JE files, flavor for unit test support.

Parameters:

suffix - which type of file we're looking for

Returns:

array of file names

getFullFileNames

java.lang.String[] getFullFileNames(long fileNum)

Returns:

the full file name and path for the nth JE file.

removeFiles

public static void removeFiles(java.io.File envFile)
                        throws java.io.IOException

Remove files from the environment home directory.

Parameters:

envFile - environment directory

Throws:

java.io.IOException

getFullFileName

public java.lang.String getFullFileName(long fileNum,
                               java.lang.String suffix)

Returns:

the full file name and path for the given file number and suffix.

getFileName

public static java.lang.String getFileName(long fileNum,
                           java.lang.String suffix)

Returns:

the file name for the nth file.

renameFile

public void renameFile(long fileNum,
              java.lang.String newSuffix)
                throws DatabaseException,
                       java.io.IOException

Rename this file to NNNNNNNN.suffix. If that file already exists, try NNNNNNNN.suffix.1, etc. Used for deleting files or moving corrupt files aside.

Parameters:

fileNum - the file we want to move

newSuffix - the new file suffix

Throws:

DatabaseException

java.io.IOException

deleteFile

public void deleteFile(long fileNum)
                throws DatabaseException,
                       java.io.IOException

Delete log file NNNNNNNN.

Parameters:

fileNum - the file we want to move

Throws:

DatabaseException

java.io.IOException

getFileLogVersion

public int getFileLogVersion(long fileNum)
                      throws LogException,
                             DatabaseException

Returns the log version for the given file.

Throws:

LogException

DatabaseException

getFileHandle

FileHandle getFileHandle(long fileNum)
                   throws LogException,
                          DatabaseException

Return a read only file handle that corresponds the this file number. Retrieve it from the cache or open it anew and validate the file header. This method takes a latch on this file, so that the file descriptor will be held in the cache as long as it's in use. When the user is done with the file, the latch must be released.

Parameters:

fileNum - which file

Returns:

the file handle for the existing or newly created file

Throws:

LogException

DatabaseException

getFileHeaderPrevOffset

long getFileHeaderPrevOffset(long fileNum)
                       throws java.io.IOException,
                              DatabaseException

Returns:

the prevOffset field stored in the file header.

Throws:

java.io.IOException

DatabaseException

getPrevEntryOffset

long getPrevEntryOffset()

Returns:

the file offset of the last LSN that was used. For constructing the headers of log entries. If the last LSN that was used was in a previous file, or this is the very first LSN of the whole system, return 0.

bumpLsn

boolean bumpLsn(long size)

Increase the current log position by "size" bytes. Move the prevOffset pointer along.

Parameters:

size - is an unsigned int

Returns:

true if we flipped to the next log file.

writeLogBuffer

void writeLogBuffer(LogBuffer fullBuffer)
              throws DatabaseException

Write out a log buffer to the file.

Parameters:

fullBuffer - buffer to write

Throws:

DatabaseException

readFromFile

void readFromFile(java.io.RandomAccessFile file,
                java.nio.ByteBuffer readBuffer,
                long offset,
                long fileNo)
            throws DatabaseException,
                   java.io.IOException

Read a buffer from a file at a given offset, using NIO if so configured.

Throws:

DatabaseException

java.io.IOException

syncLogEnd

void syncLogEnd()
          throws DatabaseException

FSync the end of the log.

Throws:

DatabaseException

syncLogEndAndFinishFile

void syncLogEndAndFinishFile()
                       throws DatabaseException,
                              java.io.IOException

Sync the end of the log, close off this log file. Should only be called under the log write latch.

Throws:

DatabaseException

java.io.IOException

groupSync

void groupSync()
         throws DatabaseException

Flush a file using the group sync mechanism, trying to amortize off other syncs.

Throws:

DatabaseException

clear

public void clear()
           throws java.io.IOException,
                  DatabaseException

Close all file handles and empty the cache.

Throws:

java.io.IOException

DatabaseException

close

public void close()
           throws java.io.IOException,
                  DatabaseException

Clear the file lock.

Throws:

java.io.IOException

DatabaseException

lockEnvironment

public boolean lockEnvironment(boolean readOnly,
                      boolean exclusive)
                        throws DatabaseException

Lock the environment. Return true if the lock was acquired. If exclusive is false, then this implements a single writer, multiple reader lock. If exclusive is true, then implement an exclusive lock. There is a lock file and there are two regions of the lock file: byte 0, and byte 1. Byte 0 is the exclusive writer process area of the lock file. If an environment is opened for write, then it attempts to take an exclusive write lock on byte 0. Byte 1 is the shared reader process area of the lock file. If an environment is opened for read-only, then it attempts to take a shared lock on byte 1. This is how we implement single writer, multi reader semantics. The cleaner, each time it is invoked, attempts to take an exclusive lock on byte 1. The owning process already either has an exclusive lock on byte 0, or a shared lock on byte 1. This will necessarily conflict with any shared locks on byte 1, even if it's in the same process and there are no other holders of that shared lock. So if there is only one read-only process, it will have byte 1 for shared access, and the cleaner can not run in it because it will attempt to get an exclusive lock on byte 1 (which is already locked for shared access by itself). If a write process comes along and tries to run the cleaner, it will attempt to get an exclusive lock on byte 1. If there are no other reader processes (with shared locks on byte 1), and no other writers (which are running cleaners on with exclusive locks on byte 1), then the cleaner will run.

Throws:

DatabaseException

releaseExclusiveLock

public void releaseExclusiveLock()
                          throws DatabaseException

Throws:

DatabaseException

checkEnvHomePermissions

public boolean checkEnvHomePermissions(boolean readOnly)
                                throws DatabaseException

Ensure that if the environment home dir is on readonly media or in a readonly directory that the environment has been opened for readonly access.

Returns:

true if the environment home dir is readonly.

Throws:

DatabaseException

truncateLog

public void truncateLog(long fileNum,
               long offset)
                 throws java.io.IOException,
                        DatabaseException

Truncate a log at this position. Used by recovery to a timestamp utilities and by recovery to set the end-of-log position.

This method forces a new log file to be written next, if the last file (the file truncated to) has an old version in its header. This ensures that when the log is opened by an old version of JE, a version incompatibility will be detected. [#11243]

Throws:

java.io.IOException

DatabaseException

forceNewLogFile

void forceNewLogFile()

Set the flag that causes a new file to be written before the next write.

firstLogEntryOffset

public static int firstLogEntryOffset()

Returns:

the size in bytes of the file header log entry.

getNextLsn

public long getNextLsn()

Return the next available LSN in the log. Note that this is unsynchronized, so is only valid as an approximation of log size.

getLastUsedLsn

public long getLastUsedLsn()

Return the last allocated LSN in the log. Note that this is unsynchronized, so if it is called outside the log write latch it is only valid as an approximation of log size.

getNFSyncs

public long getNFSyncs()

getNFSyncRequests

public long getNFSyncRequests()

getNFSyncTimeouts

public long getNFSyncTimeouts()

loadStats

void loadStats(StatsConfig config,
             EnvironmentStats stats)
         throws DatabaseException

Throws:

DatabaseException

getCacheKeys

java.util.Set<java.lang.Long> getCacheKeys()