My favorites | Sign in
Project Home Downloads Wiki Issues Source
Search
for
Philosophy_and_Design_of_Alembic  
The design and original philosophy of the Alembic API
Updated Aug 16, 2011 by ble...@gmail.com

CONCEPTS AND PHILOSOPHY IN ALEMBIC INTRO

This document is intended for developers who wish to understand so of the original thinking and design philosophy in Alembic so that they can see why things are organized they way they are in the library. This is not an end-user’s guide for things like the reference Maya or PRMan plugins.

The Alembic system is intended to facilitate baked (cached) geometry workflows and platform-independent geometry interchange and sharing. It accomplishes this by being, at its core, a library for efficiently storing samples of hierarchically related typed data. Layered on top of that is a system for interpreting that data as geometry or geometric transforms, but it’s important to keep in mind the primacy of data management at its center. This document aims to acquaint the reader with the core concepts in Alembic, and how they are used to create geometric representations.

SECTION 1: Layers of the Library

Alembic itself is composed of several layers of libraries, each of which builds extensively on the layer below it. Starting with the lowest layer and moving up towards concrete implementation, there is:

Alembic::AbcCoreAbstract (abbreviated namespace: “AbcA”)

The AbcCoreAbstract library is an almost-pure abstract library that describes the interfaces that the storage layer (e.g. AbcCoreHDF5) has to implement for Alembic. The AbcCoreAbstract layer specifies the interface for concepts like Objects or Properties, and the byte-size of data. More on the Alembic concepts like Objects or Properties in Section 2. Almost all argument and return types in this layer are boost::shared_ptrs, and their names will have the suffix “Ptr” to reflect that. Boost shared pointers provides a powerful means for managing resources without a lot of manual work on your part and this makes the entire Alembic library easier to use and more consistent with respect to memory management.

One important piece of Alembic that is defined in this layer is Alembic’s notion of time, in the form of the TimeSampling and TimeSamplingType classes. These are non-abstract classes, and provide a rich interface for working with the temporal component of data samples, and are some of the only classes from this layer that are directly and prominently exposed in higher layers.

AbcCoreAbstract is not intended to be the primary human-friendly data-manipulation library in Alembic; that distinction is held by the Abc library.

Alembic::Abc (abbreviated namespace: “Abc”)

The Abc library, as mentioned above, is the human-friendly data-manipulation library in Alembic, and provides object-based wrappers around the shared pointers from the AbcCoreAbstract layer. As well, and in contrast with AbcCoreAbstract, the Abc layer provides for interpretation of the data manipulated with it, eg, rather than a shared pointer to a contiguous block of 1024 bytes at the AbcCoreAbstract layer, at the Abc layer those bytes can be thought of as a value instance of an Imath::M44d.

Alembic::AbcGeom (abbreviated namespace: “AbcGeom”)

The AbcGeom library builds atop Abc, and is intended to be the primary layer for dealing with data with a geometric interpretation, eg, PolyMesh or Xform. Although Alembic is separated into layers as described here, all lower namespaces are lifted into each namespace above it, so that when client code has a statement like “using namespace AbcGeom;” all the namespaced identifiers from AbcA or Abc, as well as from AbcGeom, are available without qualification.

A note about writing and reading data

Because Alembic is intended as a caching system, and not a live scenegraph, the API is split into two roughly symmetrical halves: one for writing, and one for reading. At the Abc and AbcGeom layers, classes that start with ‘O’ are for writing (or “output”), and classes that start with ‘I’ are for reading (or “input”). This is analogous to the C++ iostreams conceptual separation of istreams and ostreams.

SECTION 2: Container Hierarchy in Alembic

As above, so below.

The notion of hierarchy is central to Alembic, and that is reflected in the structure of containers in Alembic. For purposes of this section, all the classes here are from the Abc layer, unless otherwise noted with an abbreviated namespace qualifier, eg, “AbcA::chrono_t”. As well, the containers discussed here are given without their ‘O’ or ‘I’ prefixes, unless it’s important to distinguish between writing and reading (eg, rather than talking about an Abc::OCompoundProperty, I will say merely “CompoundProperty”).

That said, here is the high-level overview of the container hierarchy in Alembic. Each container will be discussed in greater detail once their overall relationships are established.

Archive

An Alembic Archive is the top-level container, and is in practice the C++ class representing the actual on-disk file. Archives contain Objects.

Object

An Alembic Object is the main unit of hierarchy in Alembic. You can think of an Alembic Archive as a typical UNIX-style filesystem (eg, ext2fs), and of Alembic Objects as directories in that filesystem. The analogy is not perfect, as will be seen shortly, but it’s not bad. Objects don’t contain data directly, but instead provide structure for filling with more directly-data-containing entities. Alembic Objects are the primary unit of encapsulation. Objects contain Properties.

Property

There are two types of Alembic properties, Simple and Compound. A CompoundProperty is used to hold properties. It is the main type of container you’ll be working with. Simple Properties hold samples and can be either Scalar or Array. These samples are what actually holds your data.

Scalar Property

An Alembic ScalarProperty is a Simple Property that contains Samples whose type and number of elements (the extent) is fixed and known prior to writing. Examples of ScalarProperties are FloatProperty (each Sample is a 32-bit floating point number; extent = 1), StringProperty (each sample is a single string, of whatever size; extent = 1), or an M44dProperty (each Sample is sixteen 64-bit floating point numbers; extent = 16). The maximum extent for a ScalarProperty is 256.

Array Property

An Alembic ArrayProperty is a Simple Property that contains Samples whose type is fixed and known prior to writing, but whose extent is variable. Examples of ArrayProperties are DoubleArrayProperty (each Sample is an array of varying length, each array element being a single 64-bit floating point number), V3fArrayProperty (each Sample is an array of varying length, and each element in the array being an Imath::Vec3f, which is three 32-bit floating point numbers), or M44fArrayProperty (each Sample is an array of varying length, and each array element is an Imath::M44f, or sixteen 32-bit floating point numbers).

Sample

An Alembic Sample is the container that composes raw data and a time into a single entity. As with Properties, Samples are either Scalar or Array.

To Recap:

Archives contain Objects; Objects contain Properties, which can be either Compound or Simple; Compound Properties contain Properties, which can be Simple or Compound; Simple Properties contain Samples, and can be Scalar or Array; Samples contain data.

There are some subtleties about the above statement, but those will be elucidated in code samples later on.

SECTION 3: Time

Because Alembic is a sampling system, and does not natively store things like animation curves or natively provide interpolated values, there is a rich system for recording and recovering the time information associated with the data stored in it. In the classes AbcA::TimeSamplingType and AbcA::TimeSampling, you will find that interface.

TimeSamplingType

The TimeSamplingType class controls how properties in Alembic relate time values to their sample indices.

The default behavior is where there is a time value associated with sample zero, and a uniform time amount between each subsequent sample. This is Uniform time sampling, and would correspond to sampling every frame at 1/24 per second, or similar. Without an explicit interval between samples, it defaults to 1.0 units of time (time is specified in the Alembic C++ code with the chrono_t type, which is typedefed in AbcCoreAbstract to a 64-bit floating point number).

The second behavior is where there is a period of time over which a fixed number of samples are distributed unevenly - imagine a render scene sampled across a shutter period at shutter-begin-open, shutter-full-open, shutter-begin-close, shutter-full-close. This is Cyclic time sampling.

The final behavior is where the time samples follow no repeating scheme, though time must progress in a strictly increasing manner. This is called Acyclic time sampling.

In the most general case, instances of TimeSamplingType classes are constructed by supplying two numbers: an unsigned 32-bit integer (uint32_t) that represents the number of samples per cycle, and a signed 64-bit floating point number (chrono_t) that represents the time per cycle. However, there are convenience constructors for Uniform time sampling:

    TimeSamplingType()
     
: m_numSamplesPerCycle( 1 )
       
, m_timePerCycle( 1.0 ) {}

   
explicit TimeSamplingType( chrono_t iTimePerCycle )
     
: m_numSamplesPerCycle( 1 )
     
, m_timePerCycle( iTimePerCycle ) {}

TimeSampling

The TimeSampling class is the main interface to time information. It’s constructed in general with an instance of TimeSamplingType, and an array of chrono_ts of length TimeSamplingType::m_numSamplesPerCycle. If the TimeSamplingType instance is not Acyclic, then all times beyond startTime + timePerCycle are calculated rather than stored, though this is transparent to any client code. There is a convenience constructor in the case of Uniform TimeSamplingType:

    TimeSampling( chrono_t iTimePerCycle, chrono_t iStartTime );

When accessing time information, that information is returned as a std::pair<index_t, chrono_t>. The typename “index_t” is typedefed in AbcCoreAbstract to be a signed 64-bit integer, and is used for sample indices; see lib/Alembic/AbcCoreAbstract/Foundation.h The TimeSampling class has four main methods for getting time information:

    chrono_t getSampleTime( index_t iIndex ) const;

   
//! Find the largest valid index that has a time less than or equal
   
//! to the given time. Invalid to call this with zero samples.
   
//! If the minimum sample time is greater than iTime, index
   
//! 0 will be returned.
    std
::pair<index_t, chrono_t> getFloorIndex( chrono_t iTime,
                                                 index_t iNumSamples
) const;

   
//! Find the smallest valid index that has a time greater
   
//! than the given time. Invalid to call this with zero samples.
   
//! If the maximum sample time is less than iTime, index
   
//! numSamples-1 will be returned.
    std
::pair<index_t, chrono_t> getCeilIndex( chrono_t iTime,
                                                index_t iNumSamples
) const;

   
//! Find the valid index with the closest time to the given
   
//! time. Invalid to call this with zero samples.
    std
::pair<index_t, chrono_t> getNearIndex( chrono_t iTime,
                                                index_t iNumSamples
) const;

You’ll note that getSampleTime() takes only an index, from which an actual time value is computed using the scheme specified in the TimeSampling instance’s TimeSamplingType member. The three getFooIndex() methods, though, also take a number of samples, and will use that to clamp the returned index to iNumSamples - 1.

The reason for the return type being the pair that it is is to allow you to check your requested time against the time associated with that sample index. The following relationship will always be true:

getFloorIndex (someTime, numSamples).second ==
   getSampleTime
(getFloorIndex(someTime).first, numSamples);

Not just for getFloorIndex(), but for getCeilIndex() and getNearIndex() as well.

With these four methods, I hope it is clear how client code can implement interpolators.

TimeSampling instances are stored in an array in the Archive. When constructing Simple Properties for writing, an index into that array can be provided to the Property’s constructor, so that Samples given to the Property can have the appropriate time information associated. If no such index is given, the default TimeSampling will be assumed, which is Uniform with start time of 0.0, and 1.0 chrono_t per cycle.

SECTION 4: Deep dive into Simple Properties

Remember the container hierarchy: Simple Properties contain Samples; Samples contain data. For most purposes, though, Samples and data can be considered as the same thing, and most client code is going to deal with creating and manipulating Properties in some way. In this section, we’ll be dealing with the Abc level, which is data-centric, rather than geometric. In a later section, we’ll show how the AbcGeom layer uses the Properties from Abc to articulate a geometric interpretation of data.

OScalarProperties

First off, we’ll be focusing on the ‘O’ side of the API, for writing. At the Abc layer, the Simple Properties are further broken into typed and untyped, as well as Scalar vs. Array. The untyped OScalarProperties, as defined in lib/Alembic/Abc/OScalarProperty.h, differ mainly from the OTypedScalarProperty in that they accept data for sampling in the form of a void*, and they must be constructed with an instance of AbcA::DataType, a class that describes the size and type of the primitive data in a Sample, so that Alembic can accurately keep track of resources. An OTypedScalarProperty, on the other hand, can infer the size and type of Samples given to it because the class itself knows what it’s supposed to hold. Most client code will be dealing with TypedScalarProperties, but since an OTypedScalarProperty is-an OScalarProperty, some of the methods and members of OTypedScalarProperties will be coming from the parent class. This section is going to focus on the OTypedScalarProperties.

Looking at the constructors for an OTypedScalarProperty, the main one looks like this:

    //! Create a new TypedScalarProperty
   
//! as a child of the passed COMPOUND_PTR
   
//! Arguments can specify metadata, timesampling, and error handling.
   
template <class COMPOUND_PTR>
   
OTypedScalarProperty(
         COMPOUND_PTR iParent
,
         
const std::string &iName,

         
const Argument &iArg0 = Argument(),
         
const Argument &iArg1 = Argument(),
         
const Argument &iArg2 = Argument() );

The template argument, COMPOUND_PTR, is any type that can be intrusively converted to an AbcA::CompoundPropertyWriterPtr (“intrusively converted” meaning the type either has-a or is-a CompoundPropertyWriterPtr that can be extracted from the concrete type, or cast to from a derived type, respectively) to be used as the parent in a parent-child hierarchical relationship. The second argument is a string to be used as the name of the Property. The final three optional arguments, of type Abc::Argument, are boost::variants, which is a union-esque multi-type class, and will be explained shortly, but the comment gives a clue about what you can pass in there. Because Argument is a multi-type, you may pass in as many or as few optional arguments as you’d like, without worrying about their position in the ctor call.

Don’t be alarmed if the preceding paragraph didn’t make a lot of sense; there’s a lot of context specific to Alembic whose meaning is only implied. Let’s break it down:

  • one of the obligatory arguments, iParent, is the parent (duh) of this Property: all entities constructed by Alembic client code must be constructed hierarchically, except for the Archive itself;
  • the parent of a Simple Property is always a Compound Property;
  • the name given in the second argument must be unique at that level of hierarchy;
  • Simple Properties are constructed with an optional reference to a TimeSampling instance;
  • the types in Abc are object-based wrappers around the more primitive types from AbcCoreAbstract; an OCompoundProperty has-an AbcA::CompoundPropertyWriterPtr.

Those are the Big Ideas for Simple Properties. There are a couple Small Ideas, though, less specific to Simple Properties, and more generic to Alembic as a whole:

  • the suffix “Ptr”, as in AbcA::CompoundPropertyWriterPtr, means that the type is a boost::shared_ptr to an instance of the suffix’s root (in that case, an AbcA::CompoundPropertyWriter);
  • entities in Alembic may be constructed with metadata; specifically, with an instance of AbcA::MetaData, which is a string to string key-value hashmap.

In practice, Simple Properties for writing, not just OScalarProperties, have as their parent an Abc::OCompoundProperty.

Anyway, this is beginning to digress from OScalarProperties! Back on track!

If you look in the file lib/Alembic/Abc/OTypedScalarProperty.h, at the bottom of the file, there are a bunch of typedefs of templated OTypedScalarProperties to some meaningful name, eg:

typedef OTypedScalarProperty<BooleanTPTraits>         OBoolProperty;
typedef OTypedScalarProperty<Uint8TPTraits>           OUcharProperty;
typedef OTypedScalarProperty<Int8TPTraits>            OCharProperty;
typedef OTypedScalarProperty<Uint16TPTraits>          OUInt16Property;
typedef OTypedScalarProperty<Int16TPTraits>           OInt16Property;
typedef OTypedScalarProperty<Uint32TPTraits>          OUInt32Property;
typedef OTypedScalarProperty<Int32TPTraits>           OInt32Property;
typedef OTypedScalarProperty<Uint64TPTraits>          OUInt64Property;
typedef OTypedScalarProperty<Int64TPTraits>           OInt64Property;
typedef OTypedScalarProperty<Float16TPTraits>         OHalfProperty;
typedef OTypedScalarProperty<Float32TPTraits>         OFloatProperty;
typedef OTypedScalarProperty<Float64TPTraits>         ODoubleProperty;
typedef OTypedScalarProperty<StringTPTraits>          OStringProperty;
typedef OTypedScalarProperty<WstringTPTraits>         OWstringProperty;

etc. The template argument, the various “TPTraits”, are exactly analogous to STL type traits, and specify things like how many bytes each datum in the Property will take, their interpretation, etc. These type traits are defined in the file lib/Alembic/Abc/TypedPropertyTraits.h, if you want to take a look. When you construct an OTypedScalarProperty, you’d never do this:

OTypedScalarProperty myProp( parent, this_is_the_property_name );

Instead, you’d invoke one of the typedefed names, like so:

ODoubleProperty myDoubleProp( parent, this_is_the_property_name );

and Alembic will know that it’s intended to hold 64-bit floating point numeric values.

Speaking of a Property holding values, the Property has a method, set(), with a signature like this:

void set( const value_type &iVal )

The type declaration of the argument, value_type, is typedefed at the top of OTypedScalarProperty.h, and is defined in the traits template argument for the Property. For an ODoubleProperty, value_type is Alembic::Util::float64_t. See lib/Alembic/Util/PlainOldDataType.h for a list of the most primitive types in Alembic.

Back to the matter at hand: storing values in a Property. Let’s say you’ve constructed an ODoubleProperty, “myDoubleProp”, as shown above. Because it was not constructed with a reference to a TimeSampling instance, it will use the default TimeSampling, and assume that data given to it starts with time 0.0, and each subsequent Sample is meant to be associated with 1.0 chrono_ts after the previous Sample. The following code will set ten Samples on myDoubleProp:

for ( size_t i = 0 ; i < 10 ; ++i )
{
   myDoubleProp
.set( 2.0 * i );
}

Now, when you call myDoubleProp.getNumSamples(), it will return 10. The 0th Sample has the value 0.0, the 1th has the value 2.0, the 2th 4.0, etc. The following software listing, if compiled (and linked with the right libs) and run, will produce an Alembic Archive that contains a single Simple Property, with the values as described above:

#include <Alembic/Abc/All.h>
// the following include is for the concrete implementation of Alembic
#include <Alembic/AbcCoreHDF5/All.h>

using namespace Alembic::Abc;

int main( int, char** )
{
   
OArchive archive( Alembic::AbcCoreHDF5::WriteArchive(), myFirstArchive.abc );
   
OObject child( archive.getTop(), childObject );
   
ODoubleProperty myDoubleProp( child.getProperties(), doubleProp );
   
for ( size_t i = 0 ; i < 10 ; ++i )
   
{
       myDoubleProp
.set( 2.0 * i );
   
}
   
return 0;
}

Ta-da! Pretty easy, right? Of course, there’s that HDF5 stuff, and the matter of archive.getTop().getProperties(), but that will become clear later. You might guess, though, that getProperties() is a method that returns an Abc::OCompoundProperty, and you’d be right. The full name of that method is Abc::OObject::getProperties(), which might lead you to also guess, correctly, that Abc::OArchive::getTop() returns an Abc::OObject. Once again, we are digressing from the discussion of OScalarProperties, but we’ll return to the matter of getTop() and getProperties() later.

As simple as this code is, it may raise more questions than it answers, aside from the questions hinted at in the preceding paragraph:

  • When does the data get written?
  • Also, what if instead of an ODoubleProperty, I had an OReallyHeavyValueTypeProperty, such that I was allocating values for passing to set() on the heap; when could I safely delete it?

Those are very good questions!

You may have noticed that there was no explicit resource management being done in this code. That’s because Alembic is doing that for you, through the use of shared pointers. In Alembic, data is written once all its enclosures have gone out of scope. So, once main() is left, the Archive writes itself out to disk, the Archive’s top Object does the same, the top Object’s Compound Property follows, and finally that Compound Property’s Simple Property (written to disk as “doubleProp”). Using a program like abcecho, from $ALEMBIC_SRC_ROOT/examples/bin/AbcEcho/AbcEcho.cpp, you can see that the resulting Archive has a single Simple Property, as we expect:

$ abcecho myFirstArchive.abc
Object name=/childObject
   
ScalarProperty name=doubleProp;interpretation=;datatype=float64_t;numsamps=10
$

As for the second good question, “when can I safely free resources?” the answer is, “As soon as you call set(), Alembic owns that data, and you may safely free it.” This is true for all Properties (scalar or array) in Alembic, as well as true for the high-level geometric types in AbcGeom.

At this point, it would be instructive to digress from the focus on writing, and expand the source listing above into a write/read example, showing how the scoping mechanism works, as well as illustrating the API symmetry.

#include <Alembic/Abc/All.h>
#include <Alembic/AbcCoreHDF5/All.h>
#include <iostream>

using namespace Alembic::Abc;

int main( int, char** )
{
   std
::string archiveName( myFirstArchive.abc );
   
{ // open new scope for writing
       
OArchive archive( Alembic::AbcCoreHDF5::WriteArchive(),
                         archiveName
);
       
OObject child( archive.getTop(), childObject );
       
ODoubleProperty myDoubleProp( child.getProperties(), doubleProp );
       
for ( std::size_t i = 0 ; i < 10 ; ++i )
       
{
           myDoubleProp
.set( 2.0 * i );
       
}
   
} // the Archive is written here when it goes out of scope

   
{ // open new scope for reading
       
IArchive archive( Alembic::AbcCoreHDF5::ReadArchive(),
                         archiveName
);
       
IObject child( archive.getTop(), childObject );
       
IDoubleProperty myDoubleProp( child.getProperties(), doubleProp );
       
for ( std::size_t i = 0 ; i < myDoubleProp.getNumSamples() ; ++i )
       
{
           std
::cout << i << th sample is << myDoubleProp.getValue( i )
                     
<< std::endl;
       
}
   
} // end of scope for reading

   
return 0;
}

Sign in to add a comment
Powered by Google Project Hosting