Introduction
Some problems, especially in numerical integration and Markov Chain Monte Carlo, benefit from transformation of variables: for example, if $σ > 0$ is a standard deviation parameter, it is usually better to work with log(σ)
which can take any value on the real line. However, in general such transformations require correcting density functions by the determinant of their Jacobian matrix, usually referred to as "the Jacobian".
Also, is usually easier to code MCMC algorithms to work with vectors of real numbers, which may represent a "flattened" version of parameters, and would need to be decomposed into individual parameters, which themselves may be arrays, tuples, or special objects like lower triangular matrices.
This package is designed to help with both of these use cases. For example, consider the "8 schools" problem from Chapter 5.5 of Gelman et al (2013), in which SAT scores $y_{ij}$ in $J=8$ schools are modeled using a conditional normal
and the $θⱼ$ are assume to have a hierarchical prior distribution
For this problem, one could define a transformation
using TransformVariables
t = as((μ = asℝ, σ = asℝ₊, τ = asℝ₊, θs = as(Array, 8)))
dimension(t)
11
which would then yield a NamedTuple
with the given names, with one of them being a Vector
:
julia> x = randn(dimension(t))
11-element Array{Float64,1}:
0.2972879845354616
0.3823959677906078
-0.5976344767282311
-0.01044524463737564
-0.839026854388764
0.31111133849833383
2.2950878238373105
-2.2670863488005306
0.5299655761667461
0.43142152642291204
0.5837082875687786
julia> y = transform(t, x)
(μ = 0.2972879845354616, σ = 1.4657923768059056, τ = 0.5501113994952206, θs = [-0.0104452, -0.839027, 0.311111, 2.29509, -2.26709, 0.529966, 0.431422, 0.583708])
julia> keys(y)
(:μ, :σ, :τ, :θs)
julia> y.θs
8-element Array{Float64,1}:
-0.01044524463737564
-0.839026854388764
0.31111133849833383
2.2950878238373105
-2.2670863488005306
0.5299655761667461
0.43142152642291204
0.5837082875687786
Further worked examples of using this package can be found in the DynamicHMCExamples.jl repository. It is recommended that the user reads those first, and treats this documentation as a reference.
General interface
TransformVariables.as
— Function.as(T, args...)
Shorthand for constructing transformations with image in T
. args
determines or modifies behavior, details depend on T
.
Not all transformations have an as
method, some just have direct constructors. See methods(as)
for a list.
Examples
as(Real, -∞, 1) # transform a real number to (-∞, 1)
as(Array, 10, 2) # reshape 20 real numbers to a 10x2 matrix
as((a = asℝ₊, b = as𝕀)) # transform 2 real numbers a NamedTuple, with a > 0, 0 < b < 1
TransformVariables.dimension
— Function.dimension(t::AbstractTransform)
The dimension (number of elements) that t
transforms.
Types should implement this method.
TransformVariables.transform
— Function.transform(t, x)
Transform x
using t
.
TransformVariables.transform_and_logjac
— Function.transform_and_logjac(t, x)
Transform x
using t
; calculating the log Jacobian determinant, returned as the second value.
TransformVariables.inverse
— Function.inverse(t::AbstractTransform, y)
Return x
so that transform(t, x) ≈ y
.
inverse(t)
Return a callable equivalen to y -> inverse(t, y)
.
TransformVariables.inverse!
— Function.inverse!(x, t::AbstractTransform, y)
Put inverse(t, y)
into a preallocated vector x
, returning x
.
Generalized indexing should be assumed on x
.
See inverse_eltype
for determining the type of x
.
TransformVariables.inverse_eltype
— Function.inverse_eltype(t::AbstractTransform, y)
The element type for vector x
so that inverse!(x, t, y)
works.
TransformVariables.transform_logdensity
— Function.transform_logdensity(t, f, x)
Let $y = t(x)$, and $f(y)$ a log density at y
. This function evaluates f ∘ t
as a log density, taking care of the log Jacobian correction.
TransformVariables.random_arg
— Function.random_arg(x; kwargs...)
A random argument for a transformation.
Keyword arguments
A standard multivaritate normal or Cauchy is used, depending on cauchy
, then scaled with scale
. rng
is the random number generator used.
TransformVariables.random_value
— Function.random_value(t; kwargs...)
Random value from a transformation.
Keyword arguments
A standard multivaritate normal or Cauchy is used, depending on cauchy
, then scaled with scale
. rng
is the random number generator used.
Specific transformations
Scalar transforms
TransformVariables.∞
— Constant.Placeholder representing of infinity for specifing interval boundaries. Supports the -
operator, ie -∞
.
TransformVariables.asℝ
— Constant.Transform to the real line (identity).
TransformVariables.asℝ₊
— Constant.Transform to a non-negative real number.
TransformVariables.asℝ₋
— Constant.Transform to a non-positive real number.
TransformVariables.as𝕀
— Constant.Transform to the unit interval (0, 1)
.
Special arrays
TransformVariables.UnitVector
— Type.UnitVector(n)
Transform n-1
real numbers to a unit vector of length n
, under the Euclidean norm.
CorrCholeskyFactor(n)
Cholesky factor of a correlation matrix of size n
.
Transforms $n×(n-1)/2$ real numbers to an $n×n$ upper-triangular matrix Ω
, such that Ω'*Ω
is a correlation matrix (positive definite, with unit diagonal).
Aggregation of transformations
FIXME explain as
syntax
Defining custom transformations
TransformVariables.logjac_forwarddiff
— Function.logjac_forwarddiff(f, x; handleNaN, cfg)
Calculate the log Jacobian determinant of f
at x
using `ForwardDiff.
Note
f
should be a bijection, mapping from vectors of real numbers to vectors of equal length.
When handleNaN = true
(the default), NaN log Jacobians are converted to -Inf.
TransformVariables.value_and_logjac_forwarddiff
— Function.value_and_logjac_forwarddiff(f, x; flatten, handleNaN, cfg)
Calculate the value and the log Jacobian determinant of f
at x
. flatten
is used to get a vector out of the result that makes f
a bijection.
CustomTransform(g, f, flatten)
Wrap a custom transform y = f(transform(g, x))
in a type that calculates the log Jacobian of
∂y/∂x
using
ForwardDiff` when necessary.
Usually, g::TransformReals
, but when an integer is used, it amounts to the identity transformation with that dimension.
flatten
should take the result from f
, and return a flat vector with no redundant elements, so that $x ↦ y$ is a bijection. For example, for a covariance matrix the elements below the diagonal should be removed.
Internals
Types for various transformations
These are not part of the API, use the as
constructor or one of the predefined constants.
Scalar transformations
TransformVariables.Identity
— Type.struct Identity <: TransformVariables.ScalarTransform
Identity $x ↦ x$.
struct ScaledShiftedLogistic{T<:Real} <: TransformVariables.ScalarTransform
Maps to (scale, shift + scale)
using x ↦ logistic(x)*scale + shift
.
TransformVariables.ShiftedExp
— Type.struct ShiftedExp{D, T<:Real} <: TransformVariables.ScalarTransform
Shifted exponential. When D::Bool == true
, maps to (shift, ∞)
using x ↦ shift + eˣ
, otherwise to (-∞, shift)
using x ↦ shift - eˣ
.
Aggregating transformations
struct ArrayTransform{T<:TransformVariables.AbstractTransform, M} <: TransformVariables.VectorTransform
Apply transformation
repeatedly to create an array with given dims
.
struct TransformTuple{K, T<:Tuple{Vararg{TransformVariables.AbstractTransform,K}}} <: TransformVariables.VectorTransform
Transform consecutive groups of real numbers to a tuple, using the given transformations.
struct TransformNamedTuple{names, T<:(Tuple{Vararg{TransformVariables.AbstractTransform,N}} where N)} <: TransformVariables.VectorTransform
Transform consecutive groups of real numbers to a named tuple, using the given transformations.
Wrapper for inverse
struct InverseTransform{T}
Inverse of the wrapped transform. Use the 1-argument version of inverse
to construct.
Types and type aliases
TransformVariables.RealVector
— Type.An AbstractVector
of <:Real
elements.
Used internally as a type for transformations from vectors.
abstract type AbstractTransform
Supertype for all transformations in this package.
Interface
The user interface consists of
dimension
transform
transform_and_logjac
- [
inverse
]@(ref),inverse!
inverse_eltype
.
abstract type ScalarTransform <: TransformVariables.AbstractTransform
Transform a scalar (real number) to another scalar.
Subtypes mustdefine transform
, transform_and_logjac
, and inverse
; other methods of of the interface should have the right defaults.
abstract type VectorTransform <: TransformVariables.AbstractTransform
Transformation that transforms <: RealVector
s to other values.
Implementation
Implements transform
and transform_and_logjac
via transform_with
, and inverse
via inverse!
.
Conditional calculation of log Jacobian determinant
TransformVariables.LogJacFlag
— Type.abstract type LogJacFlag
Flag used internally by the implementation of transformations, as explained below.
When calculating the log jacobian determinant for a matrix, initialize with
logjac_zero(flag, x)
and then accumulate with log jacobians as needed with +
.
When flag
is LogJac
, methods should return the log Jacobian as the second argument, otherwise NoLogJac
, which simply combines to itself with +
, serving as an empty placeholder. This allows methods to share code of the two implementations.
TransformVariables.LogJac
— Type.Calculate log Jacobian as the second value.
TransformVariables.NoLogJac
— Type.Don't calculate log Jacobian, return NOLOGJAC
as the second value.
TransformVariables.logjac_zero
— Function.logjac_zero(?, T)
Initial value for log Jacobian calculations.
Macros
TransformVariables.@calltrans
— Macro.Workaround for https://github.com/JuliaLang/julia/issues/14919 to make transformation types callable.
TODO: remove when this issue is closed, also possibly remove MacroTools as a dependency if not used elsewhere.
Helper functions
TransformVariables.transform_with
— Function.transform_with(flag::LogJacFlag, t::AbstractTransform, x::RealVector)
Transform elements of x
, starting using transformation
.
The first value returned is the transformed value, the second the log Jacobian determinant or a placeholder, depending on flag
.
In contrast to [transform
] and [transform_and_logjac
], this method always assumes that x
is a RealVector
, for efficient traversal. Some types implement the latter two via this method.
Implementations should assume generalized indexing on x
.
TransformVariables._transform_tuple
— Function._transform_tuple(flag, x, index)
Helper function for transforming tuples. Used internally, to help type inference. Use via transfom_tuple
only.
TransformVariables._inverse!_tuple
— Function._inverse!_tuple(x, ts, ys)
Helper function for inverting tuples of transformations. Used internally.
TransformVariables._inverse_eltype_tuple
— Function._inverse_eltype_tuple(ts, ys)
Helper function determining element type of inverses from tuples. Used internally.
TransformVariables.unit_triangular_dimension
— Function.unit_triangular_dimension(n)
Number of elements (strictly) above the diagonal in an $n×n$ matrix.
Building blocks for transformations
TransformVariables.l2_remainder_transform
— Function.(y, r, ℓ) =
l2_remainder_transform(flag, x, r)
Given $x ∈ ℝ$ and $0 ≤ r ≤ 1$, return (y, r′)
such that
$y² + r′² = r²$,
$y: |y| ≤ r$ is mapped with a bijection from
x
.
ℓ
is the log Jacobian (whether it is evaluated depends on flag
).
TransformVariables.l2_remainder_inverse
— Function.