---
# **F5 EXTENSION SPECIFICATION: Color Spaces and Image Data**
---

# 0. Introduction and Scope

This document specifies the F5 representation of color image data, color spaces, and
video sequences. It extends the F5 core specification
([F5_Layout.md](F5_Layout.md)) using the named ChartDomain mechanism established in
[F5_Extension_1_NamedTypes.md](F5_Extension_1_NamedTypes.md).

Color data is more complex than a triple of numbers. The same three numbers represent
different colors depending on the color space. A file that stores only numbers without
color space information is semantically incomplete — this is identical in structure
to storing a coordinate triple without specifying whether it is Cartesian, spherical,
or cylindrical. F5 solves it by the same mechanism: **color spaces are ChartDomains**
([F5_Extension_1_NamedTypes.md §2](F5_Extension_1_NamedTypes.md)). The color channel
names are the coordinate function names of the color space; transformation rules
between color spaces are chart transformation rules stored in the file. This makes
every F5 image file semantically self-describing with respect to color — a reader can
correctly interpret stored values without guessing from file format conventions.

**The implicit-sRGB problem:** JPEG stores RGB values with no stated color space,
relying on convention. PNG supports ICC profile embedding. Many workflows ignore color
space entirely, treating all RGB data identically and producing incorrect results when
color spaces differ. F5 eliminates this class of error by design.

**Natural coverage of video:** F5 already supports time-dependent data via the
timeslice hierarchy ([F5_Layout.md §2](F5_Layout.md)). A sequence of image Grids
across timeslices is a video. This document also covers 3D image volume datasets
for temporally compressed storage.

**Scope boundary:** Texture coordinates (UV mapping) and the relationship between
image Grids and mesh Grids are specified in the textured mesh extension, which
depends on this document.

---

# 1. The Topological Nature of a Pixel

Before specifying how images are stored, it is necessary to establish what a pixel
*is* topologically. This follows from physics, not from software convention.

## 1.1 Pixels are cells, not vertices

A photosite on a CCD sensor integrates light intensity over its physical area. The
stored value is an average over that area — not a point sample at a corner. A pixel
is therefore a **2D cell**: a face of the pixel lattice, not a vertex.

In F5 terms ([F5_Layout.md §5](F5_Layout.md)):
- **Lattice corners** are vertices: `F5::IndexDepth = 0` — no color values stored here
- **Pixels** are cells: `F5::IndexDepth = 1` — color values stored here
- Both may exist as separate Skeletons within the same Grid

An image of width W and height H has W×H pixel cells and (W+1)×(H+1) lattice
vertices. The vertex Skeleton (`F5::IndexDepth = 0`) is not normally stored unless
vertex-located data is needed — but geographic and astronomical coordinates are
*always* vertex-located (they give the positions of lattice corners, not pixel
centers), so when a geospatial or astronomical image carries coordinate information,
both Skeletons are present:

```
<ImageGrid>/
    Points/    F5::IndexDepth = 0    <- vertices: geographic/WCS coordinates
    Faces/     F5::IndexDepth = 1    <- cells: pixel color values
```

The FITS file format — the standard for astronomical imaging — treats pixel data as
cell-centered. Correct tiled rendering, resampling, and interpolation require the
distinction: adjacent tiles of cell-centered data partition the index space cleanly,
while adjacent tiles of point-sample data share boundary values (see §5.3 and
[F5_Layout.md §9.3](F5_Layout.md) for the overlap attribute mechanism).

## 1.2 The pragmatic approximation

Many workflows treat pixels as point samples at cell centers — a common approximation
that simplifies many operations. F5 supports this via `F5::IndexDepth = 0` on the
image Skeleton. This is explicitly an approximation; a reader encountering
`IndexDepth = 0` treats pixels as point samples.

Both approaches may coexist within the same file and even within the same Grid.
The IndexDepth value is the normative discriminator — no additional convention is
needed.

---

# 2. Color Spaces as ChartDomains

## 2.1 The principle and self-description goal

A color space defines a coordinate system on the color gamut manifold. The channel
names (R, G, B; L, a, b; Y, Cb, Cr) are the coordinate function names — the
"measurement instructions" per [F5_DesignPhilosophy.md §7.1](F5_DesignPhilosophy.md).
A `Description` attribute (advisory, UTF-8) and a `URL` attribute (authoritative
standard reference) SHOULD be stored on every color space ChartDomain to make files
self-describing without implicit conventions.

## 2.2 Color space domains

Color spaces are grouped by their channel name structure, which reflects the
mathematical nature of the color space:

**R, G, B domain** — tristimulus spaces encoding weighted responses to long, medium,
and short wavelengths. Members: sRGB, LinearRGB, Display P3, ACEScg, Rec.709,
Rec.2020. Different members share channel names but differ in transfer functions
and primaries. Within-domain transformations are matrix multiplications (between
linear members) or piecewise nonlinear functions.

**X, Y, Z domain** — CIE standard observer coordinates, device-independent. Used
as a universal reference intermediate. Transformation from R,G,B primaries to X,Y,Z
is an invertible 3×3 matrix — a renderer can receive XYZ data and display it via
matrix multiplication in a shader, reading the matrix from the transformation rule
stored in the file.

**L, a, b domain** — perceptually uniform spaces (CIE Lab, ICtCp). Transformations
from X,Y,Z involve cube-root operations.

**Y, Cb, Cr domain** — luma-chroma spaces for video compression. Derived from R,G,B
via linear matrix and range scaling.

## 2.3 Precision variants

Color spaces commonly require precision variants beyond the geometric
SinglePrecision/DoublePrecision convention — which itself is merely a naming
convention, not a restriction (geometric ChartDomains are equally unrestricted):

- `Uint8` — 8-bit unsigned integer; standard for display-ready images
- `Uint16` — 16-bit unsigned integer; high-bit-depth, medical, scientific
- `Float16` — half-precision float; GPU texture formats, HDR
- `Float32` — single-precision float; HDR, linear light computations
- `Float64` — double-precision float; scientific measurement
- Mixed types — e.g. RGB as `Uint8` but alpha as `Float32`

The precision variant name is a label only. A reader inspects the committed type's
primitive type and size via the HDF5 API directly.

---

# 3. Defined Color Space ChartDomains

This list is non-exhaustive. Applications define additional ChartDomains following
the same pattern.

## 3.1 sRGB

IEC 61966-2-1. Non-linear gamma-encoded. Most common for JPEG and web content.

```
/Charts/sRGB/
    Uint8/
        Point     struct { uint8 R, uint8 G, uint8 B }
                    Attribute ChartDomain:   "sRGB"
                    Attribute URL:           "https://www.iec.ch/publication/6169"
                    Attribute Description:   "IEC 61966-2-1. Gamma-encoded RGB.
                                             Uint8 maps [0,255] to [0.0,1.0]."
    Uint16/
        Point     struct { uint16 R, uint16 G, uint16 B }
    Float32/
        Point     struct { float R, float G, float B }
    Point         soft link -> Uint8/Point
```

## 3.2 sRGBA

sRGB with alpha. Alpha is always linear (not gamma-encoded).

```
/Charts/sRGBA/
    Uint8/
        Point     struct { uint8 R, uint8 G, uint8 B, uint8 A }
                    Attribute Description: "sRGB gamma-encoded RGB with linear alpha.
                                           A: 0=transparent, 255=opaque."
    Float32/
        Point     struct { float R, float G, float B, float A }
    Point         soft link -> Uint8/Point
```

## 3.3 LinearRGB

Linear-light RGB. Working space for physically based rendering and compositing.
Integer variants are not defined — linear values below 1/255 are lost in 8-bit.

```
/Charts/LinearRGB/
    Float32/
        Point     struct { float R, float G, float B }
                    Attribute Description: "Linear light RGB. No gamma encoding.
                                           Standard working space for rendering."
    Float64/
        Point     struct { double R, double G, double B }
    Point         soft link -> Float32/Point
```

## 3.4 Display P3

Wide-gamut color space for modern displays. Same gamma as sRGB, larger gamut.

```
/Charts/DisplayP3/
    Uint8/
        Point     struct { uint8 R, uint8 G, uint8 B }
                    Attribute URL: "https://www.dcimovies.com/specification"
    Float32/
        Point     struct { float R, float G, float B }
    Point         soft link -> Uint8/Point
```

## 3.5 ACEScg

Academy Color Encoding System. Linear light, very wide gamut. Standard for VFX.

```
/Charts/ACEScg/
    Float16/
        Point     struct { float16 R, float16 G, float16 B }
                    Attribute URL: "https://docs.acescentral.com/specifications/acescg"
    Float32/
        Point     struct { float R, float G, float B }
    Point         soft link -> Float32/Point
```

## 3.6 CIE XYZ

Device-independent reference. All color spaces are transformable to/from CIE XYZ.

```
/Charts/CIEXYZ/
    Float32/
        Point     struct { float X, float Y, float Z }
                    Attribute URL: "https://cie.co.at/publications/colorimetry-4th-edition"
    Float64/
        Point     struct { double X, double Y, double Z }
    Point         soft link -> Float32/Point
```

## 3.7 CIE Lab

Perceptually uniform. L* encodes lightness; a*, b* encode chromatic opposition.

```
/Charts/CIELab/
    Float32/
        Point     struct { float L, float a, float b }
                    Attribute Description: "L* in [0,100]; a*, b* in [-128,127] typical."
    Point         soft link -> Float32/Point
```

## 3.8 YCbCr

Luma-chroma encoding for video compression and broadcast.

```
/Charts/YCbCr/
    Uint8/
        Point     struct { uint8 Y, uint8 Cb, uint8 Cr }
                    Attribute Description: "Y luma [16,235]; Cb/Cr chroma [16,240].
                                           Rec. ITU-R BT.601 studio swing."
    Float32/
        Point     struct { float Y, float Cb, float Cr }
    Point         soft link -> Uint8/Point
```

## 3.9 Grayscale

Single-channel luminance.

```
/Charts/Grayscale/
    Uint8/     Point  struct { uint8 L }
    Uint16/    Point  struct { uint16 L }
    Float32/   Point  struct { float L }
    Point      soft link -> Uint8/Point
```

---

# 4. Color Space Transformation Rules

## 4.1 Structure: transformation as a single dataset

A color space transformation from space A to space B is stored as a **single dataset**
directly at the path `/Charts/A/B` — not as a group. A dataset, not a group, because
there is exactly one transformation object per direction. A description attribute may
be added directly to that dataset. This is consistent with F5's principle of encoding
information in structure rather than in keywords
([F5_DesignPhilosophy.md Axiom 2](F5_DesignPhilosophy.md)).

The reverse transformation from B to A is stored at `/Charts/B/A`. Both may exist;
neither is required if the transformation is derivable from the other analytically.

## 4.2 Piecewise power-law transfer functions

Transfer functions (gamma curves) are stored as an array of segments with an
anonymous compound type — directly evaluable without string parsing, forward-compatible
to any number of segments.

Segment structure (anonymous compound type):
```
{ double threshold,  <- upper bound of this segment in source space
                     <- use IEEE +inf for the final unbounded segment
  double scale,      <- multiplicative coefficient
  double exponent,   <- power exponent (1.0 = linear segment)
  double offset      <- additive offset: C_out = scale * C_in^exponent + offset
}
```

A reader iterates segments in ascending threshold order and applies the formula for
the first segment whose threshold exceeds the input value.

**sRGB → LinearRGB** (complete IEC 61966-2-1 specification, two segments):
```
/Charts/sRGB/
    LinearRGB    Dataset {2}  anonymous compound {threshold,scale,exponent,offset}
                   row 0: { 0.04045,    1/12.92,  1.0,  0.0           }
                   row 1: { +inf,       1/1.055,  2.4,  -0.055/1.055  }
                   Attribute Description: "IEC 61966-2-1 sRGB to linear decode.
                                           Piecewise: linear below 0.04045,
                                           power law above."
```

**LinearRGB → sRGB**:
```
/Charts/LinearRGB/
    sRGB         Dataset {2}  anonymous compound {threshold,scale,exponent,offset}
                   row 0: { 0.0031308,  12.92,    1.0,  0.0    }
                   row 1: { +inf,       1.055,    1/2.4, -0.055 }
```

This two-segment array encodes the complete sRGB standard unambiguously. The same
structure accommodates other piecewise power-law transfer functions (Rec.709, Rec.2020)
by changing the coefficients, without any structural modification. A reader evaluates
the coefficients procedurally — no format version change needed for new color spaces
of this family.

## 4.3 Matrix transformations (linear color spaces)

Transformations between linear color spaces are 3×3 matrix multiplications. The
destination color space is readable from the path:

```
/Charts/LinearRGB/
    CIEXYZ    Dataset {3,3}  type: double
              <- columns: R, G, B primaries in XYZ; transforms LinearRGB -> CIEXYZ
              Attribute Description: "IEC 61966-2-1 linearized sRGB primaries to CIE XYZ D65."
```

The matrix is invertible; the reverse transform is at `/Charts/CIEXYZ/LinearRGB`.
A renderer displaying XYZ data as RGB reads this matrix and applies it in the shader.

## 4.4 LUT-based transformations

For color spaces without simple analytic forms, a 3D lookup table encodes the
transformation. The dataset type encodes the destination color space — making the
destination machine-readable without any additional metadata:

```
/Charts/ACEScg/
    sRGB    Dataset {N,N,N}  type: sRGB/Float32/Point
            <- N³ entries; ACEScg coordinates index the LUT
            <- output type: sRGB/Float32/Point encodes destination space
            <- N derivable from dataset dimension (commonly N=33)
            Attribute Description: "ACEScg to sRGB via 33³ tetrahedral LUT."
```

## 4.5 Fallback for non-piecewise-power functions

Transfer functions not expressible as piecewise power-law (SMPTE ST 2084/PQ for HDR,
HLG for hybrid log-gamma) use a `type` string attribute as a pragmatic fallback:

```
/Charts/ST2084/
    LinearRGB    Attribute: type  "ST2084_PQ"
```

This is an acknowledged pragmatic exception to the structural encoding principle
([F5_DesignPhilosophy.md Axiom 2](F5_DesignPhilosophy.md)). Future extensions may
introduce structural encodings for these cases; the per-field versioning mechanism
([F5_Performance_TableOfContents.md §3](F5_Performance_TableOfContents.md)) enables
such upgrades without breaking existing files.

---

# 5. Image Grid Structure

## 5.1 The image Grid

An image is stored as a Grid containing one or more Skeletons with color Fields.
Following §1, the image Skeleton has `F5::SkeletonDimensionality = 2` and
`F5::IndexDepth = 1` (cell-centered, topologically correct) or `= 0` (point-sample
approximation). The Skeleton name carries no normative meaning
([F5_DesignPhilosophy.md Axiom 2](F5_DesignPhilosophy.md)) — only the identity
triple (IndexDepth, SkeletonDimensionality, Refinement) identifies it.

```
/t=0/
    <ImageName>/                     <- Grid

        Charts/
            <ColorSpace>/
                GlobalChart          -> /Charts/<ColorSpace>/

        Faces/                       <- Skeleton (or Points for IndexDepth=0)
            F5::SkeletonDimensionality   2
            F5::IndexDepth               1    <- cell-centered (topologically correct)

            <ColorSpace>/            <- color Representation
                Color                <- type: <ColorSpace>/Point
                                     <- TypeInfo: DirectProduct
                                     <- Dataset {height, width}
```

Image dimensions are implicit in the dataset dimensions — not stored as explicit
attributes ([F5_Layout.md §11.2](F5_Layout.md)). This implements the simplicity
axiom ([F5_DesignPhilosophy.md Axiom 3](F5_DesignPhilosophy.md)).

The color dataset (and the entire image Grid) may reside in an external HDF5 file
referenced via an HDF5 external link or Virtual Dataset (VDS). F5 provides only
the metadata layer; HDF5 provides the access mechanism. This is particularly
relevant when images are large or managed separately from the simulation data.

**Warning:** Replacing the Color dataset with a different resolution works correctly
only when that Skeleton has exactly one Field, or when all Fields on that Skeleton
are updated consistently. When multiple Fields share the Skeleton, all must be updated
together (see §5.2).

## 5.2 Multiple color Fields on the same Skeleton

Images sharing the same resolution share the same index space. Per the simplicity
axiom ([F5_DesignPhilosophy.md Axiom 3](F5_DesignPhilosophy.md)), they should be
Fields on the same Skeleton rather than separate Grids:

```
Faces/   IndexDepth=1, SkeletonDimensionality=2
    sRGB/
        DiffuseColor       Dataset {H,W}  type: sRGB/Uint8/Point
        SpecularColor      Dataset {H,W}  type: sRGB/Uint8/Point
        EmissiveColor      Dataset {H,W}  type: sRGB/Uint8/Point
    Grayscale/
        AmbientOcclusion   Dataset {H,W}  type: Grayscale/Float32/Point
        Roughness          Dataset {H,W}  type: Grayscale/Float32/Point
```

A new Grid is introduced only when images have genuinely different index spaces
(different resolutions). If future resolution changes are anticipated for some images
but not others, placing those images in separate Grids from the start — even at
identical current resolution — is justified by anticipated future complexity. Images
may also reside in external files while sharing the same Grid metadata structure;
resolution consistency across external files is then the application's responsibility.

## 5.3 Fragment overlap for tiled images

Large images stored as tiles use the `offset`, `NoOverlapStart`, and `NoOverlapEnd`
fragment attributes defined in [F5_Layout.md §9.3](F5_Layout.md).

For cell-centered pixels (`IndexDepth = 1`), adjacent tiles partition the index space
cleanly: `NoOverlapStart = {0,0}`, `NoOverlapEnd = {0,0}`. No boundary duplication.

For point-sample pixels (`IndexDepth = 0`), adjacent tiles share boundary sample
values. A tile that duplicates the last row/column of its left/top neighbour uses
`NoOverlapStart = {1,1}`. The reader uses the effective index range to avoid
double-counting:
```
effective_start[d] = offset[d] + NoOverlapStart[d]
effective_end[d]   = offset[d] + size[d] - NoOverlapEnd[d]
```

The overlap width may differ between start and end boundaries and between dimensions.
Higher overlaps support higher-order filtering kernels applied per tile.

## 5.4 Mipmap hierarchies as Skeleton refinement

Pre-filtered reduced-resolution versions (mipmaps) are Skeletons at different
refinement levels within the same Grid — reusing the AMR refinement mechanism from
[F5_Layout.md §18](F5_Layout.md). Skeleton names are irrelevant; only the Refinement
attribute identifies the level. Mipmaps may be built on vertex Skeletons
(`IndexDepth = 0`) or cell Skeletons (`IndexDepth = 1`):

```
Faces_L0/  F5::Refinement {1,1}  Color  Dataset {1024,1024}
Faces_L1/  F5::Refinement {2,2}  Color  Dataset {512,512}
Faces_L2/  F5::Refinement {4,4}  Color  Dataset {256,256}
```

The overlap attributes apply to mipmap fragments as well — particularly relevant
when filtering kernels require boundary data from adjacent tiles at the same
mipmap level.

## 5.5 Fragmented images

Large images are stored as fragmented fields with spatial tile fragments:

```
Faces/
    sRGB/
        Color     TypeInfo: FragmentedContiguous
            Tile00    Dataset {256,256}  Attribute: offset={0,0}
            Tile01    Dataset {256,256}  Attribute: offset={0,256}
            ...
```

Fragments may reside in external HDF5 files or in HDF5 Virtual Datasets — transparent
at the HDF5 level. F5 provides the semantic metadata; HDF5 provides data access.

## 5.6 Geographic and astronomical coordinate Representations

Images with geographic or astronomical coordinates carry a vertex Skeleton
(`F5::IndexDepth = 0`) alongside the cell Skeleton, because geographic and
astronomical coordinates are defined at lattice corners (vertices), not at pixel
centers. The full specification of geographic coordinate domains — covering coordinate
reference systems, EPSG codes, WCS, and projection parameters — is deferred to a
forthcoming geospatial coordinate domain extension. That extension will reference
this document.

---

# 6. Video

## 6.1 Frame sequences

A video is a time sequence of image Grids across timeslices — no new mechanism needed:

```
/t=0.000/  <ImageGrid>/Faces/sRGB/Color   <- frame 0  (new dataset)
/t=0.033/  <ImageGrid>/Faces/sRGB/Color   <- frame 1  (symbolic link = unchanged)
/t=0.067/  <ImageGrid>/Faces/sRGB/Color   <- frame 2  (new dataset)
```

Static frames use HDF5 symbolic links (same object identity = time-independent per
[F5_Layout.md §10](F5_Layout.md)). Partial time-dependency at the fragment level is
natural: a static background tile stored once, a dynamic region with a new dataset
per frame.

## 6.2 Image volumes for temporal compression

A sequence of frames may be stored as a 3D dataset where the third dimension is time:

```
Faces/
    sRGB/
        ColorVolume    Dataset {n_frames, height, width}
                       TypeInfo: DirectProduct or Contiguous
```

An image volume covers the time interval from its associated timeslice to the next.
Time steps may be non-equidistant; image volumes may differ in frame count. They
may also be fragmented both spatially and temporally.

This approach enables HDF5-level compression (DEFLATE, SZIP, Blosc) over temporal
blocks — exploiting inter-frame redundancy analogously to MPEG keyframe/delta
encoding, without a specialized codec. For scientific image sequences with
structured, low-entropy inter-frame differences, the compression benefit may be
significant. For natural video, this approach does not implement motion compensation
and will not match H.264 or AV1 compression ratios.

---

# 7. Summary of Normative Elements

| Element | Status |
|---|---|
| Pixels as cells: IndexDepth=1, topologically correct (§1.1) | Normative |
| Pixels as point samples: IndexDepth=0, pragmatic approximation (§1.2) | Permitted |
| Geographic/astronomical coordinates on vertex Skeleton (IndexDepth=0) (§1.1) | Normative |
| Color spaces as ChartDomains with channel names as coordinate functions (§2.1) | Normative |
| Description and URL attributes on color space ChartDomains (§2.1) | Recommended |
| Precision variants: Uint8/Uint16/Float16/Float32/Float64/mixed (§2.3) | Normative |
| Piecewise power-law transfer function as anonymous compound array (§4.2) | Normative |
| Transformation as single dataset at /Charts/A/B, no group (§4.1) | Normative |
| LUT dataset type encodes destination color space (§4.4) | Normative |
| Fallback type string for non-piecewise functions (§4.5) | Permitted |
| Image dimensions implicit in dataset dimensions (§5.1) | Normative |
| Same-resolution images as Fields on same Skeleton (§5.2) | Recommended |
| Overlap attributes per F5_Layout.md §9.3 for tiled images (§5.3) | Normative |
| Mipmap levels as Skeleton refinement hierarchy (§5.4) | Normative pattern |
| Fragment tiles may reside in external files or HDF5 VDS (§5.5) | Informational |
| Video as time sequence of image Grids (§6.1) | Normative |
| Image volumes for temporal compression (§6.2) | Recommended pattern |

---

# 8. Literature

**Color space standards:**
- IEC 61966-2-1: sRGB (1999). https://www.iec.ch/publication/6169
- SMPTE ST 2065-1: ACES (2012). https://docs.acescentral.com/specifications/acescg
- Rec. ITU-R BT.709 (2015); Rec. ITU-R BT.2020 (2015).
- SMPTE ST 2084: PQ HDR transfer function (2014).

**Pixel topology:**
- FITS standard (cell-centered pixel convention): Hanisch et al., A&A 376 (2001).
  https://fits.gsfc.nasa.gov/fits_standard.html
- Smith, A.R.: "A Pixel Is Not A Little Square." Microsoft Technical Memo 6 (1995).
  https://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf

**ICC profiles:** http://www.color.org
**OpenColorIO:** https://opencolorio.org
**HDF5 Virtual Datasets:** https://docs.hdfgroup.org/hdf5/develop/group___d_a_p_l.html

**F5 data model:**
Benger, W.: PhD thesis, FU Berlin / ZIB, 2004.
https://www.fiberbundle.net/papers/TensorFieldViz.pdf

---

## Usage Restriction

This specification and all documents in the F5 specification series are published for
academic and personal use. Use of this specification, or any implementation derived
from it, by employees of military or defense-related organizations, or within
facilities producing weapons or conducting research on weapon design, or for any other
military purpose, is explicitly prohibited and contrary to the intent of the authors.

This restriction follows the spirit and terms of the light++ license under which the
original F5 reference implementation was published. The rationale is stated there
directly: software is technology, technology conveys power, and the inventor bears
responsibility for deciding to whom that power is granted. This specification was
developed to advance scientific understanding — not to enable harm.

See: https://www.fiberbundle.net/doc/copyright.html

---

# **End of Document**

---