RUNTIME VALUE CONVERSIONS

Runtime memory and collection

Reclaim unreachable Runtime values between calls, or opt into collection while generated functions execute.

Value ownership

Each zero-initialized PC owns the allocations behind its Runtime values. Copying a PV copies a handle, not its backing storage. By default, values remain allocated until pv_destroy. C callers can reclaim unreachable values with pv_collect, or opt into threshold collection after completed calls with pv_call_managed, or during generated execution with pv_call_auto.

Collection is precise and non-moving. It follows supported value references and reclaims unreachable cycles as well as ordinary discarded values. Live strings, objects, and container storage keep their addresses. Released capacity becomes available to subsequent Runtime allocations in that same context.

Collection boundary

int pv_collect(PC *context, size_t root_count,
               const PV *roots, size_t *reclaimed_bytes);

Call this only when the context is idle, between completed pv_call or pv_call_kw operations. Supply every PV your C code will still need, or register its storage as described below. These retained values are called roots. Supply NULL for roots only when root_count is zero. The optional reclaimed_bytes pointer receives the released allocation bytes, including allocation metadata.

A return of 1 means collection completed; 0 means it was refused and no arena allocation was freed. Refusal leaves the existing error and execution/file counters unchanged, records the reason in GC statistics, and writes zero to reclaimed_bytes if supplied. Check the return value. A context with an error should be destroyed after recording that error.

Collection is opt-in. Ordinary pv_call calls preserve arena lifetime. Managed calls collect only after returning. The auto-call API below also collects at generated statement and iteration boundaries. None retries a failed allocation. Keep using pv_destroy when the context is finished.

Complete caller example

Convert this source using Runtime and save the output as memory-example.c.

def make_batch(seed):
    return [str(seed + i) for i in range(20)]

def main():
    print(make_batch(0)[0])
    return 0

This source prints 0 when built with the standalone entry wrapper. The following C caller instead makes 1,000 calls in one context. It retains the first batch and each current batch, allowing previous discarded batches to be reclaimed.

#include "memory-example.c"

int main(void) {
    PC context = {0};
    int status = 1;
    PV saved = pv_none();
    /* pv_fn1 is make_batch for this exact example. */
    PV function = pv_top_function(&context, pv_fn1);
    if (context.error) goto done;

    for (int i = 0; i < 1000; ++i) {
        PV argument = pv_i(i);
        PV batch = pv_call(&context, function, 1, &argument);
        if (context.error) goto done;
        if (i == 0) saved = batch;
        PV roots[2] = {saved, batch};
        if (!pv_collect(&context, 2, roots, NULL)) {
            fputs("Collection refused\n", stderr);
            goto done;
        }
    }
    PV first = pv_getitem(&context, saved, pv_i(0));
    if (context.error) goto done;
    pv_print_ex(&context, 1, &first, pv_none(), pv_none());
    if (!context.error) status = 0;
done:
    if (context.error) fprintf(stderr, "%s\n", context.error);
    pv_destroy(&context);
    return status;
}

gcc -std=c11 memory-caller.c -o memory-caller -lm

Save the caller as memory-caller.c, compile only that file, and omit PYCF_RUNTIME_MAIN. Run ./memory-caller or .\memory-caller.exe in Windows PowerShell. Expected output is 0. Find the generated function symbol in your own conversion details; do not assume another source uses pv_fn1 for the same function.

Registered roots and managed calls

int pv_roots_attach(PC *context, PRoot *registration,
                    size_t count, const PV *values);
int pv_roots_detach(PC *context, PRoot *registration);
PV pv_call_managed(PC *context, PV function, size_t argc,
                   PV *argv, size_t threshold_bytes);
PV pv_call_managed_kw(PC *context, PV function, size_t argc, PV *argv,
                      size_t nkw, const char **names, PV *values,
                      size_t threshold_bytes);

Initialize each registration with {0} and initialize its value slots before attaching. Attach returns 1 on success and 0 for invalid, duplicate, busy, errored or over-limit registration. A registration and its nonempty PV array are borrowed C storage: keep both at stable addresses until successful detach or context destruction. Use only valid values belonging to that context. Replace slots between calls as your retained values change. Do not edit registration fields directly or copy an attached registration.

After a successful completed managed call, collection runs if allocated arena bytes reach the supplied threshold. Zero disables it; a threshold above PYCF_RUNTIME_BYTES is an error before execution. The current callable, supplied positional/keyword values and returned value survive that collection. Register every other value you need across calls, including earlier results and dynamic callables. The return value is not permanently registered. Registration is also honored by explicit pv_collect.

This alternate caller uses the same memory-example.c source above. Save it separately as memory-managed.c; compile that file alone without PYCF_RUNTIME_MAIN. It retains the first batch while threshold collection reclaims discarded batches. Expected output is 0.

#include "memory-example.c"

int main(void) {
    PC context = {0};
    PRoot registration = {0};
    PV saved = pv_none();
    int status = 1;
    PV function = pv_top_function(&context, pv_fn1);
    if (!pv_roots_attach(&context, &registration, 1, &saved)) goto done;
    for (int i = 0; i < 1000; ++i) {
        PV argument = pv_i(i);
        PV batch = pv_call_managed(&context, function, 1, &argument, 65536);
        if (context.error) goto done;
        if (i == 0) saved = batch;
        PGCStats stats = pv_gc_stats(&context);
        if (stats.refusals) {
            fprintf(stderr, "Collection refused: %s\n",
                    pv_gc_status_name(stats.last_status));
            goto done;
        }
    }
    PV first = pv_getitem(&context, saved, pv_i(0));
    if (context.error) goto done;
    pv_print_ex(&context, 1, &first, pv_none(), pv_none());
    if (!context.error) status = 0;
done:
    if (context.error) fprintf(stderr, "%s\n", context.error);
    pv_destroy(&context); /* Also clears registration while it is still alive. */
    return status;
}

Detach returns 1 when it removes and clears the registration; 0 leaves it attached. Detach before the C storage leaves scope, or destroy the context while that storage is still alive. Detaching does not free the value immediately. Setting a slot to pv_none() releases that root for the next collection. An attached slot does not transfer ownership of its C storage to the runtime.

Collection during generated execution

PV pv_call_auto(PC *context, PV function, size_t argc,
                PV *argv, size_t threshold_bytes);
PV pv_call_auto_kw(PC *context, PV function, size_t argc, PV *argv,
                   size_t nkw, const char **names, PV *values,
                   size_t threshold_bytes);

Use these idle-context entry points to enable collection at generated statement and loop/comprehension boundaries as well as after the call. Local and captured cells, return values, evaluation temporaries and suspended generated callers are traced automatically. The current callable, supplied inputs and result are protected for this call; register other C-retained values using PRoot. Zero disables collection, and a threshold above the arena budget is rejected. The threshold triggers collection; it is not a cap on live memory.

Native helper callbacks and their descendants, class construction, expanded calls, module initialization and active file scopes defer collection. A single statement or native operation can still exceed the arena budget before its next safe point. There are no allocation retries or GC-triggered file closes. Ordinary calls retain their existing behavior.

Convert this source using Runtime and save it as active-memory.c. Both its standalone main and the C caller below print 1999000.

def accumulate(count):
    total = 0
    for i in range(count):
        temporary = [str(i) * 20, i]
        total += temporary[1]
    return total

def main():
    print(accumulate(2000))
    return 0

Save this caller as active-caller.c and compile that file alone with gcc -std=c11 active-caller.c -o active-caller -lm, without PYCF_RUNTIME_MAIN. Use the generated symbol from your own conversion details; pv_fn1 identifies accumulate only in this exact example.

#include "active-memory.c"

int main(void) {
    PC context = {0};
    PV function = pv_top_function(&context, pv_fn1);
    PV count = pv_i(2000);
    PV result = pv_call_auto(&context, function, 1, &count, 16384);
    int status = 1;
    if (!context.error && pv_gc_stats(&context).collections) {
        pv_print_ex(&context, 1, &result, pv_none(), pv_none());
        if (!context.error) status = 0;
    }
    if (context.error) fprintf(stderr, "%s\n", context.error);
    pv_destroy(&context);
    return status;
}

For a standalone build, use gcc -std=c11 -DPYCF_RUNTIME_MAIN -DPYCF_RUNTIME_GC_THRESHOLD=16384 active-memory.c -o active-memory -lm. The threshold macro defaults to zero. Lower thresholds can increase collection work; choose them using measured workload statistics.

Temporary roots use separate bounded storage, reused until context destruction. PYCF_RUNTIME_TEMP_ROOTS defaults to 16,384 entries. Exhausting that limit or failing to grow the buffer is a terminal MemoryError. Frame tables use the C stack under the existing local/capture and call-depth limits. This auxiliary memory is separate from the arena cap.

Memory statistics and refusal reasons

PGCStats stats = pv_gc_stats(&context) reports live_bytes, peak_bytes, completed collections, cumulative reclaimed_bytes, refusals, registered_values, last_reclaimed and last_status. Active mode also reports temporary_roots, root_storage_bytes, peak_root_storage_bytes, safe_points and deferred_points. Peak root storage includes both buffers during growth; it excludes C stack frame tables. Arena byte counts include allocation metadata; they exclude collector scratch, C root storage and other process memory. They are not process RSS. Cumulative counters saturate at SIZE_MAX.

pv_gc_status_name(stats.last_status) describes not-run, collected, busy-context, context-error, invalid-or-excess-roots, scratch-allocation-failure, invalid-object-graph, disabled or below-threshold. A refused managed collection preserves the program's result and error state, so inspect these statistics as well as context.error. Successful later collection can retry after a temporary scratch failure. The hard arena budget still applies.

What stays alive

Use valid PVs belonging to the context, caller-owned storage for the roots array, and a separate caller-owned size_t for the optional byte count. Do not keep using an omitted value after collection. Do not access the same context concurrently or invoke collection from a callback or a directly called generated function body.

Limits and refusal

Explicit pv_collect is refused during active Runtime calls. Auto calls use their generated roots at safe points; module initialization, active file scopes, native callback regions, outstanding errors and terminal failures defer those points. It also refuses a missing/oversized roots array, an unrecognized heap reference, inconsistent graph metadata, or failure to allocate its temporary allocation index. A failed trace performs no sweep.

The default root limit is 4,096, configurable through PYCF_RUNTIME_ROOTS at C build time. It bounds registered slots and the explicit collection array separately. Managed calls use an explicit array of one callable, at most 64 supplied argument values and one result; reducing the root limit below that call's requirement causes collection refusal. The collector uses an iterative work list and a temporary index of one pointer per arena allocation, outside the arena. That index is bounded by PYCF_RUNTIME_BYTES; allocation failure is reported by a zero return. Large live graphs take proportionally more time; collection has no wall-clock deadline.

Allocation metadata is included in context.bytes. Collection returns arena and retained-file-buffer capacity, but does not reset step counts, call limits, cumulative file-operation counts, or handle serials. Mutable defaults that keep accumulating values remain live and can still exhaust memory. Even an auto call can reach its limits when live data grows, native work defers collection, or one operation allocates too much before its next safe point.

The internal allocation and closure structures changed. Regenerate C and rebuild callers together; do not mix runtime structure layouts from different generated versions. Existing pv_call, PV value handles, and pv_destroy usage remain supported.

See memory/work limits, file limits, and output validation and ForgeLens.