Vector Spaces

This note introduces vector spaces, one of the most fundamental structures in mathematics, and explores why they are so useful. For example,

Google search uses vector space.
Machine learning embeddings are vectors.
Error-correcting codes are vector spaces. ...

Once you start looking vectors appear everywhere.

What is a vector ?

To explain it in one simple line :

Vector is an element of vector space

Although vectors are often represented as lists of numbers, they can also be polynomials, matrices, functions, or many other mathematical objects. What makes them vectors is not their appearance, but the operations defined on them.

Vector Space

Let's begin with the familiar vector space $\mathbb R^n$ :

For example, consider vectors in $\mathbb R^4$ We define two operations on these vectors :

u = \begin{bmatrix} u_1 \\ u_2 \\ u_3 \\ u_4 \end{bmatrix} \qquad v = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ v_4 \end{bmatrix}

Here $u_i, v_i \in \mathbb R$

Vector Addition

u + v = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ u_3 + v_3 \\ u_4 + v_4 \end{bmatrix}

Scalar Multiplication

Suppose $c \in \mathbb R$ , then :

cu = \begin{bmatrix} cu_1 \\ cu_2 \\ cu_3 \\ cu_4 \end{bmatrix}

Together with the vector space axioms, these operations make $\mathbb R^n$ a vector space.

A subset $W \subset \mathbb R^n$ is called subspace if it satisfies the following properties :

If $w, w' \in W$ , then $w + w' \in W$
If $w \in W$ and $c \in \mathbb R$ , then $cw \in W$
Zero vector belongs to $W$

Defining Vector Spaces over $\mathbb F$

A vector space $V$ over a field $\mathbb F$ is a set together with two operations :

Addition : This operation takes two vectors and returns another vector in $V$ :
$V \times V \to V$
defined by
$v,w \leadsto v + w, \space \space \forall \space v,w \in V$
Scalar Multiplication : This operation takes a scalar from $\mathbb F$ and a vector from $V$ , and returns another vector in $V$ :
$\mathbb F \times V \to V$
defined by
$c, w \leadsto cw, \space c \in \mathbb F, \space w \in V$

These two operations must satisfy the collection of axioms which are defined below.

Axioms for Vector Spaces

Addition makes $V$ into a commutative group $V^+$ , with identity denoted by $0$ .
$1v = v$ , $\forall \space v \in V$ .
$0v = 0$ , $\forall \space v \in V$ .
Associative Law : $(ab)v = a(bv),$ $\forall \space a,b \in \mathbb F, v \in V$
Distributive Law : $(a + b)v = av + bv$ $a(u + v) = au + av$ $\forall \space a,b \in \mathbb F, \space \space u,v \in V$

So far, we have defined the objects and operations of a vector space. The next question is: how can we build new vectors from existing ones ?

Span

Let's define a vector $S = (v_1, v_2, ....,v_n)$ where $v_i$ is vector. Define $X$ to be a column in $\mathbb F^n$

SX = \begin{bmatrix} v_1, v_2, ....,v_n \end{bmatrix} \begin{bmatrix} x_1 \\ .. \\ .. \\ x_n \end{bmatrix} = v_1x_1 + v_2x_2 + .... + v_nx_n

The set of all vectors that are linear combinations of $S$ form a subspace of $V$ , this is known as subspace spanned by set.

This span is the smallest subspace of $V$ that contains $S$ , and it is denoted by $Span(S)$ .

\operatorname{span}(S) = \left\{ x_1v_1+x_2v_2+\cdots+x_nv_n \;\middle|\; x_1,x_2,\ldots,x_n\in\mathbb{F} \right\}.

To give a small example where we have a span which equals to the whole of vector space :

Example in $\mathbb R^3$ Consider the vectors :
$(1,0,0),(0,1,0),(0,0,1)$
And any vector $(x,y,z)$ Can be written as :
$x(1,0,0) + y(0,1,0) + z(0,0,1)$
Therefore in this case we can say that :
$span\left\{(1,0,0),(0,1,0),(0,0,1)\right\} = \mathbb R^3$

Basis

A Basis of vector space $V$ is a set $(v_1, ... v_n)$ of vector that are independent and also spans $V$ . Set $\mathbf B = (v_1, v_2, ... ,v_n)$ is a basis of $V$ if $w \in V$ can be written in a unique combination

w = v_1x_1 + .... + v_nx_n = \mathbf BX

Dimension of a finite dimensional vector space $V$ is the number of vectors in the basis, denoted by $\dim V$ .

Computing with Basis

Suppose we choose the standard basis of $\mathbb{R}^2$

\mathbf B=\{(1,0),(0,1)\}.

Now consider the vector

v=(3,2).

Since $\mathbf B$ is a basis, the vector $v$ can be written uniquely as a linear combination of the basis vectors:

v = 3(1,0) + 2(0,1).

The coefficients $3$ and $2$ are called the coordinates of $v$ with respect to the basis $\mathbf B$ .

We denote the coordinate vector of $v$ by

[v]_B= \begin{pmatrix} 3\\ 2 \end{pmatrix}.

The vector itself has not changed. We are simply recording how to construct it from the chosen basis vectors.

Now consider a different basis

\mathbf B'=\{(1,1),(1,-1)\}.

The same vector

v=(3,2)

can now be written as

v=\frac{5}{2}(1,1)+\frac{1}{2}(1,-1).

Therefore,

[v]_{B'}= \begin{pmatrix} \frac{5}{2}\\ \frac{1}{2} \end{pmatrix}.

Although the coordinates have changed, the vector itself remains exactly the same. Only its representation with respect to the chosen basis has changed.

A vector is independent of the basis, but its coordinates are not.

Vector Spaces and Linear Codes

One of the most useful applications of vector spaces is in coding theory. In a linear error-correcting code, every codeword is simply a vector over a finite field such as $\mathbb{F}_2$ . The collection of all valid codewords forms a vector space (or more precisely, a subspace of $\mathbb{F}^n$ ). Concepts such as span, basis, dimension, and coordinates become fundamental in constructing and analyzing these codes.

For example, a basis of a linear code allows every codeword to be represented uniquely as a linear combination of basis vectors. These coefficients correspond to the information being encoded, while the resulting vector is the transmitted codeword.