dksdk_ffi_c/dksdk_ffi_c.h¶

Defines

DKSDK_FFI_OK¶

A DkSDK function returned successfully.

DKSDK_FFI_ERR_QUASI_ORPHANED¶

When you, as an API user of DkSDK, receive this return code it is an error.

There are internal use cases for this return code that are not an error. Specifically, dksdk_ffi_registry_del_orphaned uses -1 to detect that an object was legitimately orphaned. Regardless, the -1 should not bubble out of dksdk_ffi_registry_del_orphaned or any other DkSDK function unless it is an error.

DKSDK_FFI_ERR_INVALID_ARGUMENT¶

An error with the way the programmer supplied an argument.

You should always treat these error exceptionally, perhaps with an assert.

DKSDK_FFI_ERR_INVALID_STATE¶

An error with the state at runtime.

DKSDK_FFI_ERR_NOT_FOUND¶

The queried interface or object was not found.

DKSDK_FFI_ERR_METHOD_EXCEPTION_RAISED¶

The last method call raised an exception.

DKSDK_FFI_UUID_PRINTF_FORMAT¶

printf format string for dksdk_ffi_uuid.

Example:

dksdk_ffi_uuid clsid = ...;
printf("The CLSID is " DKSDK_FFI_UUID_PRINTF_FORMAT ". Error code: %d",
  DKSDK_FFI_UUID_PRINTF_ARGS(clsid), 99);

Assumes Variant 1 UUIDs which are encoded in big-endian format.

DKSDK_FFI_UUID_PRINTF_ARGS(U)¶

printf format arguments for dksdk_ffi_uuid.

See DKSDK_FFI_UUID_PRINTF_FORMAT for an example.

DK_IIDSTR_DKSDK_FFI_COM¶

The interface identifier for a DkSDK FFI COM object dksdk_ffi_i_com.

Typedefs

typedef struct dksdk_ffi *(*FFI_CREATOR)(void *creation_data)¶

A creator of FFI hosts.

Typically you would call dksdk_ffi_host_create in the creator.

Param creation_data:

[in] Opaque data that can be used during the FFI creation process.

Return:

An initialized FFI host

typedef int (*OID_GENERATE)(const dksdk_ffi_uuid *clsid, dksdk_ffi_uuid *oid, void *oidGenerateData)¶

A generator of object identifiers.

Param clsid:

[in] The class id of the object being created.

Param oid:

[inout] A pointer to an existing dksdk_ffi_uuid. On success it will be populated with a new object identifier.

Param oidGenerateData:

[in] Arbitrary data pointer given to dksdk_ffi_host_create

Return:

• 0 on success

• anything else on error

typedef struct dksdk_ffi_i_unknown dksdk_ffi_i_unknown_t¶

The IUnknown object that has no private fields and three (3) standardized COM virtual methods.

Functions

DKSDK_FFI_C_EXPORT int dksdk_ffi_uuid_parse(const char *s, dksdk_ffi_uuid *uu)¶

Parse a UUID.

Parameters:

• s -- [in] The string to parse

• uu -- [out] On output *uu will be the uuid

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when the string was not a UUID.

DKSDK_FFI_C_EXPORT struct dksdk_ffi *dksdk_ffi_host_get_or_create(FFI_CREATOR creator, void *creation_data)¶

Get the existing FFI host, or if none create a new FFI host.

Thread-safe.

The FFI "host" is the language runtime that initializes the FFI library with dksdk_ffi_host_create. For example, in DkSDK FFI Java its Java native libraries will initialize the FFI library, and then other language runtimes (DkSDK FFI C, DkSDK FFI OCaml) can be loaded by Java. In that example, DkSDK FFI Java is acting as the FFI host.

Parameters:

• creator -- [in] The creator that will be called when there is no existing FFI host initialized.

• creation_data -- [in] Opaque data that can be used during the FFI creation process.

Returns:

the existing or new FFI host

DKSDK_FFI_C_EXPORT struct dksdk_ffi *dksdk_ffi_host_get()¶

Get the existing FFI host implementation.

Thread-safe.

Returns:

the existing FFI host implementation or NULL

DKSDK_FFI_C_EXPORT int dksdk_ffi_c_oid_derive_from_clsid_and_random_bytes(const dksdk_ffi_uuid *clsid, dksdk_ffi_uuid *oid, const char *bytes, size_t n)¶

An OID generator based on random bytes you supply.

The expection is that you create a function compatible with OID_GENERATE, and inside that function you draw random bytes and call dksdk_ffi_c_oid_derive_from_clsid_and_random_bytes.

You can give that OID_GENERATE function to dksdk_ffi_host_create.

Parameters:

• clsid -- [in] The class id of the object being created.

• oid -- [inout] A pointer to an existing dksdk_ffi_uuid. On success it will be populated with a new object identifier.

• bytes -- [in] Random bytes of size n which must be at least 8 bytes.

• n -- [in] The size of bytes

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT if any argument is NULL or if n is less than 8.

DKSDK_FFI_C_EXPORT int dksdk_ffi_host_create(OID_GENERATE oidGenerate, void *oidGenerateData, struct dksdk_ffi_c_heap *heapOpt, struct dksdk_ffi_c_stack *stackOpt, size_t minHostAllocBytesToReleaseGuest, struct dksdk_ffi **pffi)¶

Initialize the DkSDK FFI host.

A best-effort attempt is made to check if the FFI host has already been initialized. However that best-effort is not thread-safe. Use dksdk_ffi_host_get_or_create to initialize the DkSDK FFI host in a thread-safe manner.

Security Considerations¶

The oidGenerate generator should be securely randomly generated to protect against hash flooding unless you are testing or can guarantee no object identifiers will be created by a third party (including future versions of the code). That is because OIDs are used in hash tables. We provide hash flooding protection by generating a couple throwaway OIDs, then use those to seed a SIP hash, and then SIP hash all object identifiers before they are used in hash tables. That method breaks if the seeds, and hence oidGenerate, is not securely randomly generated.

Use dksdk_ffi_c_oid_derive_from_clsid_and_random_bytes to generate the OIDs securely once you have a secure generator of randomness. Consider using libsodium if you don't have a good secure generator.

Parameters:

• oidGenerate -- [in] A generator of object identifiers AND of randomness.

• oidGenerateData -- [in] Optional data pointer which will be passed to the oidGenerate function.

• heapOpt -- [in] A custom heap allocator. If NULL, a default malloc/calloc/realloc/free based heap allocator is used.

• stackOpt -- [in] A custom stack allocator. If specified, it is the caller's responsibility to free the custom stack allocator. If not specified (ie. NULL), a 8MB zeroed stack is created from the heap through the heapOpt parameter; the default stack will recommend switching to the heap when it is 95% occupied.

• minHostAllocBytesToReleaseGuest -- [in] Minimum number of bytes during an allocation where the guest platform lock (if any) will be released so that other guest platform threads can make progress. Using zero (0) as the value is safe but may be very inefficient. See dksdk_ffi_host_min_host_alloc_bytes_to_release_guest for tuning guidelines.

• pffi -- [out] The address of the pointer to the DkSDK FFI internal state. On success the de-referenced address *pffi will contain a pointer to the DkSDK FFI internal state. On failure the de-referenced address *pffi will be NULL.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when pffi is NULL

• DKSDK_FFI_ERR_INVALID_STATE when the host can't be initialized

DKSDK_FFI_C_EXPORT size_t dksdk_ffi_host_min_host_alloc_bytes_to_release_guest(struct dksdk_ffi *ffi)¶

Get the minimum number of bytes requested that will trigger releasing a guest platform lock during a host platform memory allocation.

Guest platforms with a platform lock include OCaml and Python. Sometimes these platforms are called "single threaded", but that is a misnomer. Several threads can use the OCaml runtime or Python interpreter, but access to the runtime or interpreter is guarded by the platform lock.

Allocations can be expensive. DkSDK FFI allocations involve not only finding the memory but zeroing out the memory. For large allocations it is helpful to let other threads use the platform (OCaml runtime or Python interpreter) while when the host platform (ex. C) memory is being allocated.

Releasing and acquire platform locks has a small cost similar to the cost of doing an atomic fetch. You would not want to release a platform lock in a tight loop where (for example) small strings are being allocated.

In general, you want to release the guest platform locks only when the cost of host allocation is substantially higher than the cost of releasing the guest platform lock, or when the cumulative benefit of unblocking many guest platform threads is substantially higher than the cost of releasing the guest platform lock,

Guidelines?

• When the host platform has non-GC dynamic memory management of the heap like with C malloc or in Rust, the cost of an allocation is low for small blocks of memory. Consider configuring the guest platform lock to release on host memory allocations over 32KiB.

• When the host platform has GC dynamic memory management like with Java, the cost of an allocation is low for any sized block of memory. But any random allocation may trigger the expensive garbage collector. Consider configuring the guest platform lock to release on host memory allocations over 1 / 1024 of the host platform's minor heap size. This percent will need tuning.

• When the host platform has reference counting dynamic memory management like with Python, the cost of an allocation is the same as non-GC allocations. There is no concern triggering a possible expensive release of referenced objects during an allocation. Consider configuring the guest platform lock to release on host memory allocations over 32KiB.

• When large numbers of threads are using the guest platform, lower the minimum thresholds.

Parameters:

ffi -- [in] The FFI pointer from dksdk_ffi_host_create

Returns:

the minimum number of bytes

DKSDK_FFI_C_EXPORT int dksdk_ffi_c_alloc_prefer_stack(struct dksdk_ffi *ffi, const struct dksdk_ffi_c_memrequest request, struct dksdk_ffi_c_memblock *memblock)¶

Allocate a memory block on the shadow stack memory (which is not the perhaps tiny C call stack) if there is space, or otherwise allocate the memory block on the heap.

Use dksdk_ffi_c_memblock_free to free the memory block. Because the memory block may come from the shadow stack, each dksdk_ffi_c_memblock_free must come from the last dksdk_ffi_c_alloc_prefer_stack allocated memory block.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• request -- [in] The requested amount and type of memory

• memblock -- [inout] On success the structure will be initialized to the details of the newly allocated memory block

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or memblock is NULL, or when request is not valid.

DKSDK_FFI_C_EXPORT struct dksdk_ffi_c_heap *dksdk_ffi_host_get_heap(struct dksdk_ffi *ffi)¶

Gets the heap associated with the DkSDK FFI host.

Parameters:

ffi -- [in] The FFI pointer from dksdk_ffi_host_create

Returns:

the stack

DKSDK_FFI_C_EXPORT struct dksdk_ffi_c_stack *dksdk_ffi_host_get_stack(struct dksdk_ffi *ffi)¶

Gets the stack associated with the DkSDK FFI host.

Parameters:

ffi -- [in] The FFI pointer from dksdk_ffi_host_create

Returns:

the stack

DKSDK_FFI_C_EXPORT int dksdk_ffi_host_destroy(struct dksdk_ffi *ffi)¶

Terminate the DkSDK FFI host.

It is undefined behavior to terminate the DkSDK FFI host twice.

Parameters:

ffi -- [in] The FFI pointer from dksdk_ffi_host_create

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi is NULL

• DKSDK_FFI_ERR_INVALID_STATE when the FFI has already been terminated (which can't be detected reliably).

DKSDK_FFI_C_EXPORT int dksdk_ffi_validate_unknown(const struct dksdk_ffi_i_com *ffiCom)¶

Validate the DkSDK FFI COM object or interface.

This function is not meant for consumers of objects.

This function is meant for testing and validation immediately after constructing an object.

DKSDK_FFI_C_EXPORT int dksdk_ffi_show(const struct dksdk_ffi *ffi, const struct dksdk_ffi_i_com *ffiCom, char *buffer, size_t buf_size)¶

Describe the DkSDK FFI COM class or instance object.

The description encoding is UTF-8 with no NUL bytes, and a NUL terminator. The description should be a noun or a noun phrase without an article (ex. "the", "an"), like: DkSDK FFI object with OID xxxxx

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• ffiCom -- [in] The DkSDK FFI COM object to be described.

• buf_size -- [in] The size of the description buffer. Must be greater than or equal to one (1).

• buffer -- [out] The UTF-8 description buffer. The last byte of the buffer will be NUL.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT if ffiCom or buffer is NULL, or if buf_size is zero (0).

• DKSDK_FFI_ERR_INVALID_STATE if the COM object is corrupt or the COM object can't be printed safely

DKSDK_FFI_C_EXPORT int dksdk_ffi_com_object(const struct dksdk_ffi *ffi, const struct dksdk_ffi_i_com *ffiCom, char *buffer, size_t buf_size)¶

Encode the DkSDK FFI COM class or instance object, and its current COM interface, as an object reference.

An object reference is 256-bit binary blob having the first 128-bits as the object id, and the last 128-bits as the interface id.

The current COM interface is whatever non-IUnknown interface was given to dksdk_ffi_query_interface to produce the passed-in ffiCom. This function will fail if dksdk_ffi_query_interface was not used to produce the object, or if dksdk_ffi_query_interface was queried for IUnknown.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• ffiCom -- [in] The non-IUnknown DkSDK FFI COM object to be described.

• buf_size -- [in] The size of the object reference buffer. Must be exactly 2 * sizeof(dksdk_ffi_uuid).

• buffer -- [out] The object reference buffer of size 2 * sizeof(dksdk_ffi_uuid).

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT if ffi, ffiCom or buffer is NULL, or if buf_size is not 2 * sizeof(dksdk_ffi_uuid).

• DKSDK_FFI_ERR_INVALID_STATE if the COM object is corrupt

DKSDK_FFI_C_EXPORT uint64_t dksdk_ffi_c_hashmap_sip(const struct dksdk_ffi *ffi, const void *data, size_t len)¶

Get the SIP hash of arbitrary data using seed material unique to the FFI.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• data -- [in] The data to be hashed

• len -- [in] The length of the data

Returns:

The 64-bit SIP hash

DKSDK_FFI_C_EXPORT int dksdk_ffi_registry_get(struct dksdk_ffi *ffi, const dksdk_ffi_uuid *clsidOrOid, struct dksdk_ffi_i_com **ppObject)¶

Gets the registered DkSDK FFI COM class or instance object.

If found, then it returns a pointer *ppObject to that interface after calling struct dksdk_ffi_i_com::f_add_ref on it.

Classes are "class" objects in DkSDK. You can use this function with iid as a CLSID and the corresponding class object will be returned.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• clsidOrOid -- [in] The pointer to the class identifier (CLSID) or object identifier to be searched.

• ppObject -- [inout] The address of the pointer to the object. On success the de-referenced address *ppObject will contain a pointer to the object. On failure the de-referenced address *ppObject will be NULL.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_NOT_FOUND when the object can't be found

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or iid or ppObject is NULL

• DKSDK_FFI_ERR_INVALID_STATE when an object was found but was not a valid DkSDK FFI COM object

DKSDK_FFI_C_EXPORT int dksdk_ffi_registry_add(struct dksdk_ffi *ffi, struct dksdk_ffi_i_com *object)¶

Adds the DkSDK FFI COM class or instance object to the registry.

Classes are "class" objects in DkSDK. You can use this function with iid as a CLSID and the corresponding class object will be registered.

If successful, the reference count on the object will be increased by calling struct dksdk_ffi_i_com::f_add_ref on it.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• object -- [in] The pointer to the COM class or instance object.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or object is NULL

• DKSDK_FFI_ERR_INVALID_STATE when there was no more memory to register the object

DKSDK_FFI_C_EXPORT int dksdk_ffi_registry_del(struct dksdk_ffi *ffi, const dksdk_ffi_uuid *clsid_or_oid)¶

Deletes the DkSDK FFI COM class or instance object from the registry.

Classes are "class" objects in DkSDK. You can use this function with iid as a CLSID and the corresponding class object will be deleted.

If successful, the reference count on the identified object will be decremented by calling struct dksdk_ffi_i_com::f_release on it, and the object will be removed from the registry.

The expected use case is that each dksdk_ffi_registry_add is balanced with this dksdk_ffi_registry_del. It is an error if the object has already been removed.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• clsid_or_oid -- [in] The pointer to the class identifier (CLSID) or object identifier (OID) to be de-registered.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_NOT_FOUND when the object can't be found.

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or clsid_or_oid is NULL

DKSDK_FFI_C_EXPORT int dksdk_ffi_registry_del_orphaned(struct dksdk_ffi *ffi, const dksdk_ffi_uuid *clsid_or_oid)¶

Deletes the DkSDK FFI COM class or instance object from the registry, but does not decrement the reference count.

Classes are "class" objects in DkSDK. You can use this function with clsid_or_oid as a CLSID and the corresponding class object will be deleted from the registry.

The expected use case is that each dksdk_ffi_registry_add will be released with dksdk_ffi_release and then this dksdk_ffi_registry_del_orphaned will remove the object from the registry. It is not an error if the object has already been removed.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• clsid_or_oid -- [in] The pointer to the class identifier (CLSID) or object identifier to be de-registered.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or iid is NULL

DKSDK_FFI_C_EXPORT int dksdk_ffi_query_interface(struct dksdk_ffi *ffi, struct dksdk_ffi_i_com *ffiCom, const dksdk_ffi_uuid *iid, void **ppInterface)¶

Query a DkSDK FFI COM object for an interface it might implement.

All interfaces are extensions of the struct struct dksdk_ffi_i_com.

If the DkSDK FFI COM object implements the interface, then it returns a pointer *ppInterface to that interface after calling struct dksdk_ffi_i_com::f_add_ref on it.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• ffiCom -- [in] The object, or interface to an object, to be queried.

• iid -- [in] The pointer to the identifier for the interface to be queried.

• ppInterface -- [out] The address of the pointer to the interface. It is the caller's responsibility to ensure the interface pointer references a structure that matches or exceeds the size of the interface. On success the de-referenced address *ppInterface will contain a pointer to the searched-for interface. On failure the de-referenced address *ppInterface will be NULL.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_NOT_FOUND when the interface can't be found

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi, iid or ffiCom or ppInterface is NULL.

DKSDK_FFI_C_EXPORT int dksdk_ffi_add_ref(struct dksdk_ffi *ffi, struct dksdk_ffi_i_com *ffiCom)¶

Increments the reference counter for the specified DkSDK FFI COM object, or the DkSDK FFI COM object that owns the specified interface.

Thread safe.

The only default consistency guarantee is that the increment is atomic. The returned reference count may not be consistent with the real reference count under thread contention.

If the ffiCom is an interface rather than an object, the interface may provide different consistency guarantees. However, the convention is that any difference to the consistency guarantee will be documented.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• ffiCom -- [in] The object, or interface to an object, with a reference counter to be incremented

Returns:

• The new reference count. However, the count may be inconsistent.

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or ffiCom is NULL.

DKSDK_FFI_C_EXPORT int dksdk_ffi_release(struct dksdk_ffi *ffi, struct dksdk_ffi_i_com *ffiCom)¶

Decrements the reference counter for the specified DkSDK FFI COM object, or the DkSDK FFI COM object that owns the specified interface.

Thread safe.

Unlike dksdk_ffi_add_ref, by default decrementing a reference counter is guaranteed to be sequentially consistent. That means you can rely that when the returned reference counter is zero, no other thread will simultaneously receive zero. You need that guarantee to release the memory+resources just once (no double-frees). However, maintaining the guarantee is expensive; it requires all a memory fence across all CPU cores.

If the ffiCom is an interface rather than an object, the interface may provide different consistency guarantees. However, the convention is that any difference to the consistency guarantee will be documented.

Parameters:

• ffi -- [in] The FFI pointer from dksdk_ffi_host_create

• ffiCom -- [in] The object, or interface to an object, with a reference counter to be decremented

Returns:

• The new reference count

• DKSDK_FFI_ERR_INVALID_ARGUMENT when ffi or ffiCom is NULL.

DKSDK_FFI_C_EXPORT int dksdk_ffi_c_log_configure(const char *confFile)¶

Configure DkSDK FFI C logging with a logging configuration file.

Must be configured before the first dksdk_ffi_host_create to have any effect.

Here is an example logging configuration file:

level=DEBUG # TRACE, DEBUG, INFO, WARN, ERROR, FATAL
autoFlush=100 # A flush interval [ms] (off if interval <= 0)
# Console Logger
logger=console
logger.console.output=stdout # stdout or stderr
# File Logger
logger=file
logger.file.filename=log.txt
logger.file.maxFileSize=0     # 1-LONG_MAX [bytes] (1 MB if size <= 0)
logger.file.maxBackupFiles=10 # 0-255

The configuration file encoding must be 8-bit extended ASCII.

The hash sign (#) indicates a comment from the hash to the end of the line.

Only the configuration keys shown in the above example are supported.

Parameters:

confFile -- logging configuration file

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_ARGUMENT when the configuration file can't be parsed

DKSDK_FFI_C_EXPORT int dksdk_ffi_c_log_configure_android(const char *tag)¶

Configure DkSDK FFI C logging for Android.

Logs will be viewable with the Android logcat viewer. Support for Android requires that the project was built using the Android NDK toolchain. An error will be returned if Android is not supported.

The tag usually identifies the class or activity where the log call occurs, like "MyActivity". If the tag is NULL, the Android default tag (the value of getprogname()) will be used. The default tag truncated to the maximum log message size, though appropriate tags should be much smaller.

Use dksdk_ffi_c_logger_setLevel to change the logging level after configuration.

Parameters:

tag -- The source of a log message. May be null.

Returns:

• DKSDK_FFI_OK on success

• DKSDK_FFI_ERR_INVALID_STATE when the library was not compiled with the Android NDK toolchain.