Class

org.apache.spark.mllib.linalg.distributed

BlockMatrix

Related Doc: package distributed

Permalink

class BlockMatrix extends DistributedMatrix with Logging

Represents a distributed matrix in blocks of local matrices.

Annotations
@Since( "1.3.0" )
Linear Supertypes
Logging, DistributedMatrix, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. BlockMatrix
  2. Logging
  3. DistributedMatrix
  4. Serializable
  5. Serializable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BlockMatrix(blocks: RDD[((Int, Int), Matrix)], rowsPerBlock: Int, colsPerBlock: Int)

    Permalink

    Alternate constructor for BlockMatrix without the input of the number of rows and columns.

    Alternate constructor for BlockMatrix without the input of the number of rows and columns.

    blocks

    The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix. If multiple blocks with the same index exist, the results for operations like add and multiply will be unpredictable.

    rowsPerBlock

    Number of rows that make up each block. The blocks forming the final rows are not required to have the given number of rows

    colsPerBlock

    Number of columns that make up each block. The blocks forming the final columns are not required to have the given number of columns

    Annotations
    @Since( "1.3.0" )
  2. new BlockMatrix(blocks: RDD[((Int, Int), Matrix)], rowsPerBlock: Int, colsPerBlock: Int, nRows: Long, nCols: Long)

    Permalink

    blocks

    The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix. If multiple blocks with the same index exist, the results for operations like add and multiply will be unpredictable.

    rowsPerBlock

    Number of rows that make up each block. The blocks forming the final rows are not required to have the given number of rows

    colsPerBlock

    Number of columns that make up each block. The blocks forming the final columns are not required to have the given number of columns

    nRows

    Number of rows of this matrix. If the supplied value is less than or equal to zero, the number of rows will be calculated when numRows is invoked.

    nCols

    Number of columns of this matrix. If the supplied value is less than or equal to zero, the number of columns will be calculated when numCols is invoked.

    Annotations
    @Since( "1.3.0" )

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def add(other: BlockMatrix): BlockMatrix

    Permalink

    Adds the given block matrix other to this block matrix: this + other.

    Adds the given block matrix other to this block matrix: this + other. The matrices must have the same size and matching rowsPerBlock and colsPerBlock values. If one of the blocks that are being added are instances of SparseMatrix, the resulting sub matrix will also be a SparseMatrix, even if it is being added to a DenseMatrix. If two dense matrices are added, the output will also be a DenseMatrix.

    Annotations
    @Since( "1.3.0" )
  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. val blocks: RDD[((Int, Int), Matrix)]

    Permalink

    The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix.

    The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix. If multiple blocks with the same index exist, the results for operations like add and multiply will be unpredictable.

    Annotations
    @Since( "1.3.0" )
  7. def cache(): BlockMatrix.this.type

    Permalink

    Caches the underlying RDD.

    Caches the underlying RDD.

    Annotations
    @Since( "1.3.0" )
  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. val colsPerBlock: Int

    Permalink

    Number of columns that make up each block.

    Number of columns that make up each block. The blocks forming the final columns are not required to have the given number of columns

    Annotations
    @Since( "1.3.0" )
  10. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  12. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  14. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  15. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean = false): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  16. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  17. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  18. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  19. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  20. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  21. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  22. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  23. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  24. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  25. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  26. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  27. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  28. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  29. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  30. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  31. def multiply(other: BlockMatrix, numMidDimSplits: Int): BlockMatrix

    Permalink

    Left multiplies this BlockMatrix to other, another BlockMatrix.

    Left multiplies this BlockMatrix to other, another BlockMatrix. The colsPerBlock of this matrix must equal the rowsPerBlock of other. If other contains SparseMatrix, they will have to be converted to a DenseMatrix. The output BlockMatrix will only consist of blocks of DenseMatrix. This may cause some performance issues until support for multiplying two sparse matrices is added. Blocks with duplicate indices will be added with each other.

    other

    Matrix B in A * B = C

    numMidDimSplits

    Number of splits to cut on the middle dimension when doing multiplication. For example, when multiplying a Matrix A of size m x n with Matrix B of size n x k, this parameter configures the parallelism to use when grouping the matrices. The parallelism will increase from m x k to m x k x numMidDimSplits, which in some cases also reduces total shuffled data.

    Annotations
    @Since( "2.2.0" )
  32. def multiply(other: BlockMatrix): BlockMatrix

    Permalink

    Left multiplies this BlockMatrix to other, another BlockMatrix.

    Left multiplies this BlockMatrix to other, another BlockMatrix. The colsPerBlock of this matrix must equal the rowsPerBlock of other. If other contains SparseMatrix, they will have to be converted to a DenseMatrix. The output BlockMatrix will only consist of blocks of DenseMatrix. This may cause some performance issues until support for multiplying two sparse matrices is added.

    Annotations
    @Since( "1.3.0" )
    Note

    The behavior of multiply has changed in 1.6.0. multiply used to throw an error when there were blocks with duplicate indices. Now, the blocks with duplicate indices will be added with each other.

  33. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  34. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  35. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  36. val numColBlocks: Int

    Permalink
    Annotations
    @Since( "1.3.0" )
  37. def numCols(): Long

    Permalink

    Gets or computes the number of columns.

    Gets or computes the number of columns.

    Definition Classes
    BlockMatrixDistributedMatrix
    Annotations
    @Since( "1.3.0" )
  38. val numRowBlocks: Int

    Permalink
    Annotations
    @Since( "1.3.0" )
  39. def numRows(): Long

    Permalink

    Gets or computes the number of rows.

    Gets or computes the number of rows.

    Definition Classes
    BlockMatrixDistributedMatrix
    Annotations
    @Since( "1.3.0" )
  40. def persist(storageLevel: StorageLevel): BlockMatrix.this.type

    Permalink

    Persists the underlying RDD with the specified storage level.

    Persists the underlying RDD with the specified storage level.

    Annotations
    @Since( "1.3.0" )
  41. val rowsPerBlock: Int

    Permalink

    Number of rows that make up each block.

    Number of rows that make up each block. The blocks forming the final rows are not required to have the given number of rows

    Annotations
    @Since( "1.3.0" )
  42. def subtract(other: BlockMatrix): BlockMatrix

    Permalink

    Subtracts the given block matrix other from this block matrix: this - other.

    Subtracts the given block matrix other from this block matrix: this - other. The matrices must have the same size and matching rowsPerBlock and colsPerBlock values. If one of the blocks that are being subtracted are instances of SparseMatrix, the resulting sub matrix will also be a SparseMatrix, even if it is being subtracted from a DenseMatrix. If two dense matrices are subtracted, the output will also be a DenseMatrix.

    Annotations
    @Since( "2.0.0" )
  43. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  44. def toCoordinateMatrix(): CoordinateMatrix

    Permalink

    Converts to CoordinateMatrix.

    Converts to CoordinateMatrix.

    Annotations
    @Since( "1.3.0" )
  45. def toIndexedRowMatrix(): IndexedRowMatrix

    Permalink

    Converts to IndexedRowMatrix.

    Converts to IndexedRowMatrix. The number of columns must be within the integer range.

    Annotations
    @Since( "1.3.0" )
  46. def toLocalMatrix(): Matrix

    Permalink

    Collect the distributed matrix on the driver as a DenseMatrix.

    Collect the distributed matrix on the driver as a DenseMatrix.

    Annotations
    @Since( "1.3.0" )
  47. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  48. def transpose: BlockMatrix

    Permalink

    Transpose this BlockMatrix.

    Transpose this BlockMatrix. Returns a new BlockMatrix instance sharing the same underlying data. Is a lazy operation.

    Annotations
    @Since( "1.3.0" )
  49. def validate(): Unit

    Permalink

    Validates the block matrix info against the matrix data (blocks) and throws an exception if any error is found.

    Validates the block matrix info against the matrix data (blocks) and throws an exception if any error is found.

    Annotations
    @Since( "1.3.0" )
  50. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  51. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  52. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Logging

Inherited from DistributedMatrix

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped