IndexedRowMatrix

Instance Constructors

new IndexedRowMatrix(rows: RDD[IndexedRow])

Alternative constructor leaving matrix dimensions to be determined automatically.
Alternative constructor leaving matrix dimensions to be determined automatically.

Annotations
@Since( "1.0.0" )
new IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)

rows
indexed rows of this matrix
nRows
number of rows. A non-positive value means unknown, and then the number of rows will be determined by the max row index plus one.
nCols
number of columns. A non-positive value means unknown, and then the number of columns will be determined by the size of the first row.

Annotations
@Since( "1.0.0" )

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def columnSimilarities(): CoordinateMatrix

Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
returns
An n x n sparse upper-triangular matrix of cosine similarities between columns of this matrix.

Annotations
@Since( "1.6.0" )
def computeGramianMatrix(): Matrix

Computes the Gramian matrix A^T A.
Computes the Gramian matrix A^T A.

Annotations
@Since( "1.0.0" )
Note
This cannot be computed on matrices with more than 65535 columns.
def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]

Computes the singular value decomposition of this IndexedRowMatrix.
Computes the singular value decomposition of this IndexedRowMatrix. Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
The cost and implementation of this method is identical to that in org.apache.spark.mllib.linalg.distributed.RowMatrix With the addition of indices.
At most k largest non-zero singular values and associated vectors are returned. If there are k such values, then the dimensions of the return will be:
U is an org.apache.spark.mllib.linalg.distributed.IndexedRowMatrix of size m x k that satisfies U'U = eye(k), s is a Vector of size k, holding the singular values in descending order, and V is a local Matrix of size n x k that satisfies V'V = eye(k).
k
number of singular values to keep. We might return less than k if there are numerically zero singular values. See rCond.
computeU
whether to compute U
rCond
the reciprocal condition number. All singular values smaller than rCond * sigma(0) are treated as zero, where sigma(0) is the largest singular value.
returns
SingularValueDecomposition(U, s, V)

Annotations
@Since( "1.0.0" )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def multiply(B: Matrix): IndexedRowMatrix

Multiply this matrix by a local matrix on the right.
Multiply this matrix by a local matrix on the right.
B
a local matrix whose number of rows must match the number of columns of this matrix
returns
an IndexedRowMatrix representing the product, which preserves partitioning

Annotations
@Since( "1.0.0" )
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def numCols(): Long

Gets or computes the number of columns.
Gets or computes the number of columns.

Definition Classes
IndexedRowMatrix → DistributedMatrix
Annotations
@Since( "1.0.0" )
def numRows(): Long

Gets or computes the number of rows.
Gets or computes the number of rows.

Definition Classes
IndexedRowMatrix → DistributedMatrix
Annotations
@Since( "1.0.0" )
val rows: RDD[IndexedRow]

indexed rows of this matrix
indexed rows of this matrix

Annotations
@Since( "1.0.0" )
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix

Converts to BlockMatrix.
Converts to BlockMatrix. Blocks may be sparse or dense depending on the sparsity of the rows.
rowsPerBlock
The number of rows of each block. The blocks at the bottom edge may have a smaller value. Must be an integer value greater than 0.
colsPerBlock
The number of columns of each block. The blocks at the right edge may have a smaller value. Must be an integer value greater than 0.
returns
a BlockMatrix

Annotations
@Since( "1.3.0" )
def toBlockMatrix(): BlockMatrix

Converts to BlockMatrix.
Converts to BlockMatrix. Creates blocks with size 1024 x 1024.

Annotations
@Since( "1.3.0" )
def toCoordinateMatrix(): CoordinateMatrix

Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.

Annotations
@Since( "1.3.0" )
def toRowMatrix(): RowMatrix

Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.

Annotations
@Since( "1.0.0" )
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package distributed

class IndexedRowMatrix extends DistributedMatrix

Instance Constructors

new IndexedRowMatrix(rows: RDD[IndexedRow])

new IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

def columnSimilarities(): CoordinateMatrix

def computeGramianMatrix(): Matrix

def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

def multiply(B: Matrix): IndexedRowMatrix

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def numCols(): Long

def numRows(): Long

val rows: RDD[IndexedRow]

final def synchronized[T0](arg0: ⇒ T0): T0

def toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix

def toBlockMatrix(): BlockMatrix

def toCoordinateMatrix(): CoordinateMatrix

def toRowMatrix(): RowMatrix

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from DistributedMatrix

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped