# Loading C Functions in Icon

Gregg M. Townsend
Department of Computer Science
The University of Arizona

www.cs.arizona.edu/icon/v951/cfuncs.htm
Adapted from Icon Analyst 36
Last updated October 27, 2009

Dynamic loading allows Icon programs to use functions coded in C without modifying the Icon system itself. The C code is compiled and placed in a library, then loaded from the library when the Icon program runs. Here is a discussion of the use and construction of such functions.

## Program library functions

The Icon program library includes an assortment of loadable Unix interfaces and special-purpose functions. Some are there for their general utility, some for illustration, and some to fill specialized needs. Here is a sampling:

> `bitcount(i)` — count the bits set in an integer
> `chmod(s, i)` — change the permissions of a file
> `fpoll(f, i)` — poll a file for input, with timeout
> `getpid()` — return the process identification number
> `kill(i1, i2)` — send a signal to a process
> `lgconv(i)` — convert a large integer to a string
> `tconnect(s, i)` — connect a file to a TCP port
> `umask(i)` — change the process permission mask

The full set of functions can be found in the library's `cfuncs` directory. Documentation and code are also available on line. These functions are available automatically to an Icon program that includes `link cfunc`. The `bitcount()` function is a good example for detailed examination.

## Loading a function

The built-in Icon function `loadfunc(libname, funcname)` loads the C function `funcname()` from the library file `libname` and returns a procedure value. If the function cannot be loaded, the program is terminated.

If `loadfunc(libname, "myfunc")` produces `p`, then

- `p(`arguments`)` calls `myfunc()` with a list of arguments
- `type(p)` returns `"procedure"`
- `image(p)` returns `"function myfunc"`
- `proc("myfunc")` returns `p`
- `proc("myfunc", 0)` fails

The following program loads the function `bitcount()` and assigns it to a global variable of the same name. Assigning it to a global variable makes it available to other procedures, although that's not needed here. The `bitcount()` function returns the number of bits that are set in the binary representation of an integer.

>     $define Library "/icon/bin/libcfunc.so"
>     global bitcount

>     procedure main()
>        local i
>        bitcount := loadfunc(Library, "bitcount")
>        every i := 250 to 260 do
>           write(i, " ", bitcount(i))
>     end

When this program is run, it lists the integers from 250 to 260 along with their bit counts:

>     250  6
>     251  7
>     252  6
>     253  7
>     254  7
>     255  8
>     256  1
>     257  2
>     258  2
>     259  3
>     260  2

## Loading from a path

Embedding a file name such as `/icon/bin/libcfunc.so` in the program is undesirable. An alternative is for the program to find the library file using information from the program environment.

The Icon library procedure `pathload(libname, funcname)` searches the set of directories given by the `FPATH` environment variable to find `libname` and load `funcname`. As is usual in Icon path searching, the current directory is searched first. If the function cannot be loaded, the program is terminated.

The `pathload()` procedure is included by linking `pathfind` from the Icon program library. Using `pathload()`, the example program becomes:

>     $define Library "libcfunc.so"
>     link pathfind
>     global bitcount

>     procedure main()
>        local i
>        bitcount := pathload(Library, "bitcount")
>        every i := 250 to 260 do
>           write(i, " ", bitcount(i))
>     end

The default `FPATH` includes the current directory and the installed Icon program library directory. To find a library located elsewhere, `FPATH` must be set explicitly before the program is run.

## Implicit function loading

It is possible to encapsulate the loading process so that the body of an Icon program is unaware that it is calling a C function. Consider this example:

>     $define Library "libcfunc.so"
>     link pathfind

>     procedure main()
>        local i
>        every i := 250 to 260 do
>           write(i, " ", bitcount(i))
>     end

>     procedure bitcount(n)
>        bitcount := pathload(Library, "bitcount")
>        return bitcount(n)
>     end

First of all, notice that there is no longer a global declaration for `bitcount`, and that the main procedure no longer calls `pathload()`. As far as the main procedure is concerned, `bitcount()` is just another procedure to call, with no special requirements. This is a nice simplification.

The new `bitcount()` procedure is a bit tricky, though. To understand it, you must know that an Icon procedure declaration creates a global variable with an initial value of that procedure. A global variable is subject to assignment.

When `main()` calls `bitcount()` for the first time, the `bitcount()` procedure loads the `bitcount()` C function from the library. The result is assigned to the global variable `bitcount`, replacing the current procedure value. Consequently, all subsequent calls to `bitcount()` use the loaded function.

The first call to `bitcount()` remains incomplete after loading the function; the bits of `n` still must be counted. So, following loading, the procedure calls `bitcount(n)`. Although this looks like a recursive call, it isn't — the call uses the current value of the global variable `bitcount`, and so it calls the loaded C function. The bits of `n` are counted and returned, completing the first call.

After the first time, calls to `bitcount()` go directly to the loaded code. The Icon procedure `bitcount()` is no longer accessible.

## Implicit library loading

The Icon program library provides an implicit loading procedure for each of the C functions in the library. Small procedures like the `bitcount()` procedure shown above are included by linking `cfunc`. Using the library interface procedure, our example now can be simplified to this:

>     link cfunc

>     procedure main()
>        local i
>        every i := 250 to 260 do
>           write(i, " ", bitcount(i))
>     end

The `link cfunc` declaration is the only hint that `bitcount()` is written in C.

## Making connections

The bit counting example doesn't really illustrate the full potential of using C functions in an Icon program. Bit counting, after all, can be done in Icon. Here's something that can't.

The library function `tconnect(host, port)` establishes a TCP connection to a specified port number on an Internet host. TCP is a communication protocol used by telnet programs, news servers, web servers, and many other network services.

The following program makes a connection to the Icon web server and writes the contents of the Icon home page — in its original HTML markup language, of course.

>     link cfunc

>     procedure main()
>        local f
>        f := tconnect("www.cs.arizona.edu", 80)
>        writes(f, "GET /icon/ HTTP/1.0\n\n")
>        flush(f)
>        seek(f, 1)
>        while write(read(f))
>     end

The `tconnect()` call establishes the connection and returns a file that is open for both input and output. The internet host `www.cs.arizona.edu` is the web server for the University of Arizona's Department of Computer Science. (Port 80 is the standard web server port number.) The program then transmits a request for the `/icon/` web page. The details of the request string are specified by the Hypertext Transfer Protocol, not discussed here.

The `flush()` call ensures that all the data is actually sent, and then the `seek()` call resets the file in preparation for a switch from output to input. In this situation `seek()` does not actually reposition the file, but it's required when switching modes.

Finally, lines are read and echoed until an end-of-file is received.

## Writing loadable C functions

Now consider the construction of library functions. Because the Icon system expects C functions to implement a certain interface, dynamic loading usually requires specially written C functions. In general, it is not possible to use an existing C function without writing an intermediate "glue" function.

C functions must deal with the data types used by the Icon run-time system, notably the "descriptors" that represent all Icon values. While an understanding of the Icon run-time system is helpful, it is possible to create useful functions by modeling them after existing library functions. Integer and string values are most easily handled.

A loadable C function has the prototype

> `int funcname(int argc, descriptor *argv)`

where `argc` is the number of arguments and `argv` is an array of argument descriptors. The first element, `argv[0]`, is used to return an Icon value, and is initialized to a descriptor for the null value. This element is not included in the count `argc`. The actual arguments begin with `argv[1]`.

If the C function returns zero, the call from Icon succeeds. A negative value indicates failure. If a positive value is returned, it is interpreted as an error number and a fatal error with that number is signalled. In this case, if `argv[0]` is non-null, it is reported as the "offending value". There is no way for a C function to suspend, and no way to indicate a null value as an offending value in the case of an error.

## Interface macros

The C file `icall.h` contains a set of macros for use in writing loadable functions. Documentation is included as comments. This file can be found in the `cfuncs` directory in the source code of the Icon program library. Alternatively, it can be loaded from the web. Macros are provided for:

- inspecting the type of an Icon value
- validating the type of an argument
- converting an Icon value into a C value
- returning a C value in Icon form
- failing or signaling an error

Most macros deal with integers or strings. Some support also is provided for handling real and file values. The macros expect the C arguments to be declared with the names of `argc` and `argv`.

## Counting bits, again

For a concrete example of a C function, consider the source code of the `bitcount()` function used earlier:

>     #include "icall.h"

>     int bitcount(int argc, descriptor *argv)
>     {
>        unsigned long v;
>        int n;
>        ArgInteger(1);
>        v = IntegerVal(argv[1]);
>        n = 0;
>        while (v != 0) {
>           n += v & 1;
>           v >>= 1;
>           }
>        RetInteger(n);
>     }

Like all loadable functions, `bitcount()` is an integer function with two parameters, `argc` and `argv`.

The `ArgInteger` macro call verifies that argument 1 is a simple integer. (Large integers are typically rejected by C functions because of the extra work involved.) If argument 1 is missing or has the wrong type, `ArgInteger` makes the function return error code 101 (integer expected or out of range).

The `IntegerVal` macro call extracts the value of the first argument.

In each pass through the `while` loop, the low order bit of `v` is extracted (`v & 1`), added to `n`, and shifted off (`v >>= 1`). When no more nonzero bits are left, the loop exits. Note that `v` is declared unsigned to ensure that only zero bits are inserted by the shift operation.

The `RetInteger` macro call returns the value of `n` as an Icon integer.

## External data values

Loadable functions can create and return data structures that are treated as an opaque `external` type by Icon. The use of external types is described separately.

## Preparing a library

To be used in an Icon program, a C function must be built and installed in a library. Compilation comes first, usually involving a command such as

> `cc –c bitcount.c`

to produce an object file `bitcount.o`. The `–c` option causes a relocatable object file to be produced instead of a stand-alone executable program. If `icall.h` is not in the current directory, an additional option may be needed to specify its location. Other options, such as optimization options, also could be specified.

A C function can be loaded only from a "shared library". Even if there is just one function, it must be placed in a library. Library names conventionally end with a `.so` suffix.

It seems that every system has a different way to create libraries, usually involving special flags to `cc` or `ld`. The shell script `mklib.sh` embodies our understanding of shared library creation. It takes one argument naming the library to be created and one or more additional arguments listing object file names. For example, the command

> `mklib.sh mylib.so bitcount.o`

creates a file `mylib.so` containing the functions read from `bitcount.o`. Like `icall.h`, `mklib.sh` is available in the program library source code or from the web.

## Summary

- Icon programs can load and call functions written in C.
- The C functions must be tailored to Icon's requirements.
- Each function must be loaded before it can be called.
- A simple Icon procedure can be used to hide the loading details.
- `pathload()` searches `FPATH` to find a function library.
- C functions can embed data in opaque `external` values.
- Some useful functions are provided in the Icon program library.