Tensor Basics Teaser

Math Linear Algebra • Applications and Advanced Linear Algebra (capstone)

Written by STEM Calculators Team

Preview tensor operations with small 2nd/3rd order tensors: addition, outer product, contraction, and a rank-1 (CP1) decomposition teaser. The graph is a tensor grid heatmap with zoom/pan and a Play animation.

Operation

Tensor order

Dimensions

I (rows)

J (cols)

—

Tolerance

Display decimals

Tensor editor

—

Inputs accept -3.5, 2e-4, fractions 7/3, and constants pi, e.

Ready

Results

Result shape

—

Summary

—

Norm / error

—

Rank-1 factors (if applicable)

—

Result

—

Tensor grid visual

Heatmap of the result tensor. Tick labels show indices. Zoom with wheel/trackpad, drag to pan, double-click to reset. Play animates reveal (2D) or cycles slices (3D).

—

Step-by-step

Choose an operation and click “Calculate”.

Rate this calculator

0.0 /5 (0 ratings)

Be the first to rate.

Your rating

Name (optional) Review (optional)

You can update your rating any time.

Theory

Tensors as “multi-index arrays” and multilinear maps

In basic linear algebra, a scalar is a number, a vector is a 1D array, and a matrix is a 2D array. A tensor generalizes these ideas: it can be represented as an array with multiple indices. For example:

0th order tensor: scalar \(a\)
1st order tensor: vector \(v_i\)
2nd order tensor: matrix \(A_{ij}\)
3rd order tensor: \(T_{ijk}\) (a “stack” of matrices)

Beyond the array view, tensors can also be understood as multilinear maps. A matrix \(A\) defines a bilinear form \(x^{\mathsf T}Ay\), while a 3rd order tensor can define a trilinear form \(\sum_{i,j,k} T_{ijk}a_i b_j c_k\). This perspective is important in physics (stress/strain tensors), geometry, and machine learning.

Addition and scaling

Tensor addition is the direct generalization of vector and matrix addition: it is componentwise and only defined for tensors of the same shape. \[ (A+B)_{i_1\cdots i_n} = A_{i_1\cdots i_n} + B_{i_1\cdots i_n}. \] Scaling is also componentwise: \((\alpha A)_{i_1\cdots i_n}=\alpha A_{i_1\cdots i_n}\). In the UI, a 3rd order tensor is edited via slices \(k\), where each slice is an \(I\times J\) matrix.

Outer product

The outer product increases order by combining objects without summing over indices. The simplest case is the outer product of two vectors \(a\in\mathbb{R}^I\) and \(b\in\mathbb{R}^J\), producing a matrix \(C=a\otimes b\) with entries \(C_{ij}=a_i b_j\).

This teaser includes a matrix–vector outer product: if \(A\in\mathbb{R}^{I\times J}\) and \(b\in\mathbb{R}^{K}\), the result is a 3D tensor \(C\in\mathbb{R}^{I\times J\times K}\) defined by \[ C_{ijk} = A_{ij}\,b_k. \] A useful way to interpret this is “each slice is a scaled copy of the matrix”: for each \(k\), \(C(:,:,k)=b_k A\).

Contraction

Contraction is the tensor analogue of a dot product or matrix multiplication: it sums over a shared index and therefore reduces order. For example, if \(T\in\mathbb{R}^{I\times J\times K}\) and \(v\in\mathbb{R}^{K}\), contracting over \(k\) produces a matrix \(M\in\mathbb{R}^{I\times J}\): \[ M_{ij} = \sum_{k=1}^{K} T_{ijk}\,v_k. \] You can read this as a weighted sum of slices: \(M = \sum_k v_k\,T(:,:,k)\). Many tensor operations used in ML (often called “mode-n products”) are contractions with vectors or matrices along a chosen axis.

Rank-1 decomposition teaser (CP rank-1)

A rank-1 matrix can be written as an outer product \(u v^{\mathsf T}\). A general matrix can be approximated by a rank-1 matrix capturing the strongest pattern: \[ A \approx \sigma\,u v^{\mathsf T}, \] where \(\sigma\) is a scale (dominant singular value) and \(u,v\) are unit vectors.

For 3rd order tensors, the analogous building block is a rank-1 tensor \(a\otimes b\otimes c\), whose entries factor into products of one-dimensional components: \[ \hat{T}_{ijk} = \lambda\,a_i b_j c_k. \] This is the basic term in a CP (CANDECOMP/PARAFAC) decomposition, where a tensor is approximated by a sum of rank-1 terms. Full tensor algebra adds concepts like tensor rank, unfoldings/matricizations, and more stable algorithms (ALS, HOSVD, etc.). This calculator uses a small “power-method-like” iteration as a teaser suitable for tiny tensors.

ML note

In modern machine learning, “tensors” are the standard data structure for batches, sequences, and multi-channel signals: examples include images (height × width × channels), videos (time × height × width × channels), and transformer activations (batch × sequence × hidden). Many deep learning layers can be viewed as contractions between tensors (dot products along chosen axes), while outer products appear in feature interactions and attention-style constructions.

Frequently Asked Questions

Why does the tensor editor use slices?

A 3rd order tensor can be viewed as a stack of matrices. Editing by slice k is a simple way to enter T(i,j,k) values.

What is the difference between outer product and contraction?

Outer product increases order and does not sum indices (C(i,j,k)=A(i,j)*b(k)). Contraction reduces order by summing over an index (M(i,j)=sum_k T(i,j,k)*v(k)).

What does rank-1 tensor mean?

A rank-1 tensor factors into a product of vectors: T(i,j,k)=λ a(i)b(j)c(k). It is the tensor analogue of a rank-1 matrix u v^T.

Is the rank-1 decomposition exact?

Not necessarily. The calculator shows a small rank-1 approximation (teaser) and reports the reconstruction error norm.

Rate this calculator

Frequently Asked Questions

Related calculators