Introduction

Introduction

Some problems, especially in numerical integration and Markov Chain Monte Carlo, benefit from transformation of variables: for example, if $σ > 0$ is a standard deviation parameter, it is usually better to work with log(σ) which can take any value on the real line. However, in general such transformations require correcting density functions by the determinant of their Jacobian matrix, usually referred to as "the Jacobian".

Also, is usually easier to code MCMC algorithms to work with vectors of real numbers, which may represent a "flattened" version of parameters, and would need to be decomposed into individual parameters, which themselves may be arrays, tuples, or special objects like lower triangular matrices.

This package is designed to help with both of these use cases. For example, consider the "8 schools" problem from Chapter 5.5 of Gelman et al (2013), in which SAT scores $y_{ij}$ in $J=8$ schools are modeled using a conditional normal

\[y_{ij} ∼ N(θⱼ, σ²)\]

and the $θⱼ$ are assume to have a hierarchical prior distribution

\[θⱼ ∼ N(μ, τ²)\]

For this problem, one could define a transformation

using TransformVariables
t = as((μ = asℝ, σ = asℝ₊, τ = asℝ₊, θs = as(Array, 8)))
dimension(t)
11

which would then yield a NamedTuple with the given names, with one of them being a Vector:

julia> x = randn(dimension(t))
11-element Array{Float64,1}:
  0.2972879845354616
  0.3823959677906078
 -0.5976344767282311
 -0.01044524463737564
 -0.839026854388764
  0.31111133849833383
  2.2950878238373105
 -2.2670863488005306
  0.5299655761667461
  0.43142152642291204
  0.5837082875687786

julia> y = transform(t, x)
(μ = 0.2972879845354616, σ = 1.4657923768059056, τ = 0.5501113994952206, θs = [-0.0104452, -0.839027, 0.311111, 2.29509, -2.26709, 0.529966, 0.431422, 0.583708])

julia> keys(y)
(:μ, :σ, :τ, :θs)

julia> y.θs
8-element Array{Float64,1}:
 -0.01044524463737564
 -0.839026854388764
  0.31111133849833383
  2.2950878238373105
 -2.2670863488005306
  0.5299655761667461
  0.43142152642291204
  0.5837082875687786

Further worked examples of using this package can be found in the DynamicHMCExamples.jl repository. It is recommended that the user reads those first, and treats this documentation as a reference.

General interface

TransformVariables.asFunction.
as(T, args...)

Shorthand for constructing transformations with image in T. args determines or modifies behavior, details depend on T.

Not all transformations have an as method, some just have direct constructors. See methods(as) for a list.

Examples

as(Real, -∞, 1)          # transform a real number to (-∞, 1)
as(Array, 10, 2)         # reshape 20 real numbers to a 10x2 matrix
as((a = asℝ₊, b = as𝕀)) # transform 2 real numbers a NamedTuple, with a > 0, 0 < b < 1
source
dimension(t::AbstractTransform)

The dimension (number of elements) that t transforms.

Types should implement this method.

source
transform(t, x)

Transform x using t.

source
transform_and_logjac(t, x)

Transform x using t; calculating the log Jacobian determinant, returned as the second value.

source
inverse(t::AbstractTransform, y)

Return x so that transform(t, x) ≈ y.

inverse(t)

Return a callable equivalen to y -> inverse(t, y).

source
inverse!(x, t::AbstractTransform, y)

Put inverse(t, y) into a preallocated vector x, returning x.

Generalized indexing should be assumed on x.

See inverse_eltype for determining the type of x.

source
inverse_eltype(t::AbstractTransform, y)

The element type for vector x so that inverse!(x, t, y) works.

source
transform_logdensity(t, f, x)

Let $y = t(x)$, and $f(y)$ a log density at y. This function evaluates f ∘ t as a log density, taking care of the log Jacobian correction.

source
random_arg(x; kwargs...)

A random argument for a transformation.

Keyword arguments

A standard multivaritate normal or Cauchy is used, depending on cauchy, then scaled with scale. rng is the random number generator used.

source
random_value(t; kwargs...)

Random value from a transformation.

Keyword arguments

A standard multivaritate normal or Cauchy is used, depending on cauchy, then scaled with scale. rng is the random number generator used.

source

Specific transformations

Scalar transforms

Placeholder representing of infinity for specifing interval boundaries. Supports the - operator, ie -∞.

source

Transform to the real line (identity).

source

Transform to a non-negative real number.

source

Transform to a non-positive real number.

source

Transform to the unit interval (0, 1).

source

Special arrays

UnitVector(n)

Transform n-1 real numbers to a unit vector of length n, under the Euclidean norm.

source
CorrCholeskyFactor(n)

Cholesky factor of a correlation matrix of size n.

Transforms $n×(n-1)/2$ real numbers to an $n×n$ upper-triangular matrix Ω, such that Ω'*Ω is a correlation matrix (positive definite, with unit diagonal).

source

Aggregation of transformations

FIXME explain as syntax

Defining custom transformations

logjac_forwarddiff(f, x; handleNaN, cfg)

Calculate the log Jacobian determinant of f at x using `ForwardDiff.

Note

f should be a bijection, mapping from vectors of real numbers to vectors of equal length.

When handleNaN = true (the default), NaN log Jacobians are converted to -Inf.

source
value_and_logjac_forwarddiff(f, x; flatten, handleNaN, cfg)

Calculate the value and the log Jacobian determinant of f at x. flatten is used to get a vector out of the result that makes f a bijection.

source
CustomTransform(g, f, flatten)

Wrap a custom transform y = f(transform(g, x))in a type that calculates the log Jacobian of∂y/∂xusingForwardDiff` when necessary.

Usually, g::TransformReals, but when an integer is used, it amounts to the identity transformation with that dimension.

flatten should take the result from f, and return a flat vector with no redundant elements, so that $x ↦ y$ is a bijection. For example, for a covariance matrix the elements below the diagonal should be removed.

source

Internals

Types for various transformations

These are not part of the API, use the as constructor or one of the predefined constants.

Scalar transformations

struct Identity <: TransformVariables.ScalarTransform

Identity $x ↦ x$.

source
struct ScaledShiftedLogistic{T<:Real} <: TransformVariables.ScalarTransform

Maps to (scale, shift + scale) using x ↦ logistic(x)*scale + shift.

source
struct ShiftedExp{D, T<:Real} <: TransformVariables.ScalarTransform

Shifted exponential. When D::Bool == true, maps to (shift, ∞) using x ↦ shift + eˣ, otherwise to (-∞, shift) using x ↦ shift - eˣ.

source

Aggregating transformations

struct ArrayTransform{T<:TransformVariables.AbstractTransform, M} <: TransformVariables.VectorTransform

Apply transformation repeatedly to create an array with given dims.

source
struct TransformTuple{K, T<:Tuple{Vararg{TransformVariables.AbstractTransform,K}}} <: TransformVariables.VectorTransform

Transform consecutive groups of real numbers to a tuple, using the given transformations.

source
struct TransformNamedTuple{names, T<:(Tuple{Vararg{TransformVariables.AbstractTransform,N}} where N)} <: TransformVariables.VectorTransform

Transform consecutive groups of real numbers to a named tuple, using the given transformations.

source

Wrapper for inverse

struct InverseTransform{T}

Inverse of the wrapped transform. Use the 1-argument version of inverse to construct.

source

Types and type aliases

An AbstractVector of <:Real elements.

Used internally as a type for transformations from vectors.

source
abstract type AbstractTransform

Supertype for all transformations in this package.

Interface

The user interface consists of

source
abstract type ScalarTransform <: TransformVariables.AbstractTransform

Transform a scalar (real number) to another scalar.

Subtypes mustdefine transform, transform_and_logjac, and inverse; other methods of of the interface should have the right defaults.

source
abstract type VectorTransform <: TransformVariables.AbstractTransform

Transformation that transforms <: RealVectors to other values.

Implementation

Implements transform and transform_and_logjac via transform_with, and inverse via inverse!.

source

Conditional calculation of log Jacobian determinant

abstract type LogJacFlag

Flag used internally by the implementation of transformations, as explained below.

When calculating the log jacobian determinant for a matrix, initialize with

logjac_zero(flag, x)

and then accumulate with log jacobians as needed with +.

When flag is LogJac, methods should return the log Jacobian as the second argument, otherwise NoLogJac, which simply combines to itself with +, serving as an empty placeholder. This allows methods to share code of the two implementations.

source

Calculate log Jacobian as the second value.

source

Don't calculate log Jacobian, return NOLOGJAC as the second value.

source
logjac_zero(?, T)

Initial value for log Jacobian calculations.

source

Macros

Workaround for https://github.com/JuliaLang/julia/issues/14919 to make transformation types callable.

TODO: remove when this issue is closed, also possibly remove MacroTools as a dependency if not used elsewhere.

source

Helper functions

transform_with(flag::LogJacFlag, t::AbstractTransform, x::RealVector)

Transform elements of x, starting using transformation.

The first value returned is the transformed value, the second the log Jacobian determinant or a placeholder, depending on flag.

In contrast to [transform] and [transform_and_logjac], this method always assumes that x is a RealVector, for efficient traversal. Some types implement the latter two via this method.

Implementations should assume generalized indexing on x.

source
_transform_tuple(flag, x, index)

Helper function for transforming tuples. Used internally, to help type inference. Use via transfom_tuple only.

source
_inverse!_tuple(x, ts, ys)

Helper function for inverting tuples of transformations. Used internally.

source
_inverse_eltype_tuple(ts, ys)

Helper function determining element type of inverses from tuples. Used internally.

source
unit_triangular_dimension(n)

Number of elements (strictly) above the diagonal in an $n×n$ matrix.

source

Building blocks for transformations

(y, r, ℓ) =
l2_remainder_transform(flag, x, r)

Given $x ∈ ℝ$ and $0 ≤ r ≤ 1$, return (y, r′) such that

  1. $y² + r′² = r²$,

  2. $y: |y| ≤ r$ is mapped with a bijection from x.

is the log Jacobian (whether it is evaluated depends on flag).

source
(x, r′) =
l2_remainder_inverse(y, r)

Inverse of l2_remainder_transform in x and y.

source