Package

serez-ai

Neural networks, autodiff, and training loops — with a Keras-like API. Build, train, and run models in a few lines of Serez Code.

Install

sz install serez-ai

Quick start

import "serez-ai"

Random.seed(42)

// XOR dataset — inputs/targets are tensors
let X = Tensor.from([[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]])
let y = Tensor.from([[0.0], [1.0], [1.0], [0.0]])

// Build the model
let model = new Sequential()
model.add(new Dense(2, 16, "relu"))
model.add(new Dense(16, 1, "sigmoid"))

// Train — fit_opt returns the per-epoch loss history
let opt     = new Adam(0.01, 0.9, 0.999)
let history = model.fit_opt(X, y, new BCE(), 1000, opt, false)

out "Final loss: {history[history.length() - 1]}"

// Predict — forward returns a tensor; read it with .get(row, col)
let pred = model.forward(Tensor.from([[1.0, 0.0]]))
out "XOR(1, 0) ≈ {pred.get(0, 0)}"   // → ~0.97

Sequential model

Sequential stacks layers one after the other. Add layers with .add(), then train with .fit_opt():

let model = new Sequential()
model.add(new Dense(input_size, output_size, activation))

// fit_opt(X, y, loss_fn, epochs, optimizer, verbose) → per-epoch loss array
let history = model.fit_opt(X, y, new MSE(), 500, new SGD(0.01), false)

// Forward pass — returns a tensor; read values with .get(row, col)
let predictions = model.forward(X)

Layers

Layer	Parameters	Use for
`Dense`	in, out, activation	Fully connected layer. Activations: relu, sigmoid, tanh, linear
`Conv2D`	in_ch, out_ch, kernel, stride, activation	2D convolution for image data
`MaxPool2D`	pool, stride	Downsamples conv feature maps
`Flatten`	—	Flattens conv output to 1D for Dense layers
`Embedding`	vocab_size, embed_dim	Token id → dense vector lookup
`LSTM`	input_size, hidden_size	Sequences, time series, text
`GRU`	input_size, hidden_size	Lighter recurrent alternative to LSTM
`MultiHeadAttention`	d_model, n_heads	Transformer-style self-attention
`LayerNorm`	d_model, eps	Normalizes activations across features
`TransformerBlock`	d_model, n_heads, d_ff	Attention + feed-forward transformer block

// Image classifier — conv → pool → flatten → dense
let model = new Sequential()
model.add(new Conv2D(1, 4, 3, 1, "relu"))  // in_ch=1, out_ch=4, 3×3 kernel, stride 1
model.add(new MaxPool2D(2, 2))             // halve the spatial size
model.add(new Flatten())                   // conv maps → 1D vector
model.add(new Dense(36, 8, "relu"))        // flattened size (out_ch × H × W) → 8
model.add(new Dense(8, 1, "sigmoid"))      // output

Optimizers

new Adam(lr, beta1, beta2)       // e.g. new Adam(0.001, 0.9, 0.999)
new SGD(lr)                      // e.g. new SGD(0.01)
new Momentum(lr, momentum)       // e.g. new Momentum(0.01, 0.9)

Need AdamW or RMSprop? They ship as low-level Autodiff steps (below) rather than optimizer classes. Use the Autodiff steps directly for full control:

// All return [new_param, new_state...]
let res = Autodiff.adamStep(param, grad, m, v, step, lr)
let res = Autodiff.adamwStep(param, grad, m, v, step, lr, wd)
let res = Autodiff.sgdStep(param, grad, velocity, lr, momentum)
let res = Autodiff.rmspropStep(param, grad, sq_avg, lr)

Loss functions

new MSE()            // Mean Squared Error — regression
new BCE()            // Binary Cross-Entropy — binary classification

MAE and multi-class CrossEntropy are available as Autodiff functions (not loss classes) — call them directly inside the training loop:

let loss = Autodiff.mseLoss(pred, target)
let loss = Autodiff.maeLoss(pred, target)
let loss = Autodiff.bceLoss(pred, target)
let loss = Autodiff.crossEntropyLoss(logits, class_indices)

Weight initialization

let w = Autodiff.xavierUniform([fan_in, fan_out])  // tanh / sigmoid
let w = Autodiff.xavierNormal([fan_in, fan_out])
let w = Autodiff.heUniform([fan_in, fan_out])       // ReLU networks
let w = Autodiff.heNormal([fan_in, fan_out])

Saving & loading weights

Save trained model weights to a .szw binary file and reload them later:

// After training — save all parameters
Autodiff.saveWeights("model.szw", [w1, b1, w2, b2])

// In production — load without retraining
let weights = Autodiff.loadWeights("model.szw")
let w1 = weights[0]
let b1 = weights[1]

Tensors

Tensors are the core data type. All operations are tracked automatically when a tape is active.

import "serez-ai"

let w = Autodiff.heNormal([4, 8])
let x = Tensor.from([1.0, 0.0, 1.0, 1.0]).reshape([1, 4])

// Forward pass — fully tracked
Autodiff.tape()
let h = x.matmul(w).relu()
let loss = Autodiff.mseLoss(h, Tensor.zeros([1, 8]))
Autodiff.backward(loss)

// Retrieve gradient
let grad = Autodiff.gradient(w)
out grad.shape()   // → [4, 8]

Activations (all tracked)

t.relu()      t.sigmoid()    t.tanh()     t.softmax()
t.gelu()      t.leaky_relu(alpha)
t.elu(alpha)  t.swish()      t.silu()     t.mish()

Gradient utilities

// Clip gradients
let clipped = Autodiff.clipGrad(grad, max_norm)
let clipped = Autodiff.clipGradNorm([g1, g2, g3], max_norm)

// Stop gradient flow
let detached = Autodiff.stopGrad(tensor)

Random

Reproducible random numbers for dataset shuffling and weight initialization:

Random.seed(42)                               // set seed for reproducibility
let n = Random.decimal()                      // decimal in [0, 1)
let i = Random.int(0, 10)                     // int in [0, 10]
let t = Random.normalTensor([128, 64], 0.0, 0.01)   // Gaussian tensor
let t = Random.uniformTensor([128, 64], -1.0, 1.0)  // Uniform tensor