tachyon.underfs.s3

Class S3UnderFileSystem

java.lang.Object

tachyon.underfs.UnderFileSystem

tachyon.underfs.s3.S3UnderFileSystem

public class S3UnderFileSystem
extends UnderFileSystem

Under file system implementation for S3 using the Jets3t library.

Nested Class Summary

Nested classes/interfaces inherited from class tachyon.underfs.UnderFileSystem

UnderFileSystem.SpaceType, UnderFileSystem.UnderFSType

Field Summary

Fields inherited from class tachyon.underfs.UnderFileSystem

mTachyonConf

Constructor Summary

Constructors

Constructor and Description
S3UnderFileSystem(String bucketName, TachyonConf tachyonConf)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	connectFromMaster(TachyonConf conf, String hostname)
void	connectFromWorker(TachyonConf conf, String hostname)
OutputStream	create(String path)
OutputStream	create(String path, int blockSizeByte)
OutputStream	create(String path, short replication, int blockSizeByte)
boolean	delete(String path, boolean recursive)
boolean	exists(String path)
long	getBlockSizeByte(String path) There is no concept of a block in S3, however the maximum allowed size of one file is currently 5 TB.
Object	getConf()
List<String>	getFileLocations(String path)
List<String>	getFileLocations(String path, long offset)
long	getFileSize(String path)
long	getModificationTimeMs(String path)
long	getSpace(String path, UnderFileSystem.SpaceType type)
UnderFileSystem.UnderFSType	getUnderFSType()
boolean	isFile(String path)
String[]	list(String path)
boolean	mkdirs(String path, boolean createParent)
InputStream	open(String path)
boolean	rename(String src, String dst)
void	setConf(Object conf)
void	setPermission(String path, String posixPerm)

Methods inherited from class tachyon.underfs.UnderFileSystem

get, get, isHadoopUnderFS, parse, providesStorage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3UnderFileSystem

public S3UnderFileSystem(String bucketName,
                 TachyonConf tachyonConf)
                  throws org.jets3t.service.ServiceException

Throws:

org.jets3t.service.ServiceException

Method Detail

getUnderFSType

public UnderFileSystem.UnderFSType getUnderFSType()

Specified by:

getUnderFSType in class UnderFileSystem

close

public void close()
           throws IOException

Specified by:

close in class UnderFileSystem

Throws:

IOException

connectFromMaster

public void connectFromMaster(TachyonConf conf,
                     String hostname)

Specified by:

connectFromMaster in class UnderFileSystem

connectFromWorker

public void connectFromWorker(TachyonConf conf,
                     String hostname)

Specified by:

connectFromWorker in class UnderFileSystem

create

public OutputStream create(String path)
                    throws IOException

Specified by:

create in class UnderFileSystem

Throws:

IOException

create

public OutputStream create(String path,
                  int blockSizeByte)
                    throws IOException

Specified by:

create in class UnderFileSystem

Throws:

IOException

create

public OutputStream create(String path,
                  short replication,
                  int blockSizeByte)
                    throws IOException

Specified by:

create in class UnderFileSystem

Throws:

IOException

delete

public boolean delete(String path,
             boolean recursive)
               throws IOException

Specified by:

delete in class UnderFileSystem

Throws:

IOException

exists

public boolean exists(String path)
               throws IOException

Specified by:

exists in class UnderFileSystem

Throws:

IOException

getBlockSizeByte

public long getBlockSizeByte(String path)
                      throws IOException

There is no concept of a block in S3, however the maximum allowed size of one file is currently 5 TB.

Specified by:

getBlockSizeByte in class UnderFileSystem

Parameters:

path - The file name

Returns:

5 TB in bytes

Throws:

IOException - this implementation will not throw this exception, but subclasses may

getConf

public Object getConf()

Specified by:

getConf in class UnderFileSystem

getFileLocations

public List<String> getFileLocations(String path)
                              throws IOException

Specified by:

getFileLocations in class UnderFileSystem

Throws:

IOException

getFileLocations

public List<String> getFileLocations(String path,
                            long offset)
                              throws IOException

Specified by:

getFileLocations in class UnderFileSystem

Throws:

IOException

getFileSize

public long getFileSize(String path)
                 throws IOException

Specified by:

getFileSize in class UnderFileSystem

Throws:

IOException

getModificationTimeMs

public long getModificationTimeMs(String path)
                           throws IOException

Specified by:

getModificationTimeMs in class UnderFileSystem

Throws:

IOException

getSpace

public long getSpace(String path,
            UnderFileSystem.SpaceType type)
              throws IOException

Specified by:

getSpace in class UnderFileSystem

Throws:

IOException

isFile

public boolean isFile(String path)
               throws IOException

Specified by:

isFile in class UnderFileSystem

Throws:

IOException

list

public String[] list(String path)
              throws IOException

Specified by:

list in class UnderFileSystem

Throws:

IOException

mkdirs

public boolean mkdirs(String path,
             boolean createParent)
               throws IOException

Specified by:

mkdirs in class UnderFileSystem

Throws:

IOException

open

public InputStream open(String path)
                 throws IOException

Specified by:

open in class UnderFileSystem

Throws:

IOException

rename

public boolean rename(String src,
             String dst)
               throws IOException

Specified by:

rename in class UnderFileSystem

Throws:

IOException

setConf

public void setConf(Object conf)

Specified by:

setConf in class UnderFileSystem

setPermission

public void setPermission(String path,
                 String posixPerm)
                   throws IOException

Specified by:

setPermission in class UnderFileSystem

Throws:

IOException