KL Implementation

As mentioned previously, the map-reduce model is available in KL. Using the map-reduce model directly from within KL allows for simple multithreading directly from within KL itself, without the need to create structures within the host language.

KL Constant Producers

In KL, a constant value producer is created using the createConstValue(value) call. The value parameter can be any typed KL r-value, eg. a constant value, the value of a variable, or the result of a function call. The result of the value is a value of type ValueProducer<ValueType>, where ValueType is the type of value. For example, the KL code:

operator entry() {
  ValueProducer<Scalar> vp = createConstValue(1.4142);
  report(vp.produce());
}

produces the result:

1.4142

A constant array producer is created using the createConstArray(array) function. Its single parameter array must be an expression that resolves to a fixed-length, variable-length or external array. The return value is of type ArrayProducer<ElementType>, where ElementType is the type of the array elements. For example, the KL code:

operator entry() {
  String a[];
  a.push('zero'); a.push('one'); a.push('two');
  ArrayProducer<String> ap = createConstArray(a);
  report(ap.produce());
}

produces the result:

["zero","one","two"]

KL Generators

A value generator in KL is created using the createValueGenerator(functionName). functionName must be the name of a KL function available in the same KL module that has the operator signature functionName(io ValueType value); the result is a value of type ValueProducer<ValueType>. For example, the KL code:

operator gen(io String value[]) {
  value.push("Hello, world!");
}

operator entry() {
  ValueProducer<String[]>vp = createValueGenerator(gen);
  report(vp.produce());
}

produces the output:

["Hello, world!"]

An array generator in KL is created using the createArrayGenerator(countValueProducer, functionName). The countValueProducer parameter must be a value producer that produces a value of type Size, ie. a value of type ValueProducer<Size>. If you are generating a fixed-size array of size 16, for instance, you can pass the result of createConstValue(16), but you can also pass a more complex value producer such as a value generator. functionName is the name of a function in the same module with one of the following prototypes:

operator functionName(io ElementType element);
operator functionName(io ElementType element, Index index);
operator functionName(io ElementType element, Index index, Size count);

In cases where the index parameter is present, the index of the element within the array is passed to the generator function; similarly, when the count parameter is present the total number of elements in the array is passed. This can be useful for figuring out what value to generate.

An example of an array generator:

operator gen(
  io Scalar v,
  Index i,
  Size n
  )
{
  // Produces n uniform values on the interval [0,1], including 0 and 1 themselves
  v = Scalar(i) / Scalar(n-1);
}

operator entry() {
  ArrayProducer<Scalar> ap = createArrayGenerator(createConstValue(Size(10)), gen);
  report(ap.produce());
}

produces:

[0,0.1111111,0.2222222,0.3333333,0.4444444,0.5555556,0.6666667,0.7777778,0.8888889,1]

In both cases, you can optionally specify a shared value producer as the last parameter to the create...Generator() call. When a shared value producer is provided, the function receives an additional parameter whose type is the value type of the shared value producer. So, for value generators, the prototype of the operator becomes:

operator gen(io ValueType value, SharedType sharedValue) {
  ...
}

and for array generators the prototype of the operator becomes:

operator gen(io ElementType element, Index index, Size count, SharedType sharedValue) {
  ...
}

Note that when using a shared value with an array generator you must include the index and count parameters in the operator even if they are unused.

KL Maps

A value map in KL is created using the createValueMap(inputValueProducer, functionName). inputValueProducer must be a value producer, and functionName must be the name of a KL function available in the same KL module that has the operator signature functionName(InputType input, io OutputType output); the type InputType must be the same as the value type of the input value producer. The result of the createValueMap call is a value of type ValueProducer<OutputType>. For example, the KL code:

operator map(String input, io Size output) {
  output = input.length;
}

operator entry() {
  ValueProducer<Size> vp = createValueMap(createConstValue("Hello, world!"), map);
  report(vp.produce());
}

produces the output:

13

An array map in KL is created using the createArrayMap(inputArrayProducer, functionName). The inputArrayProducer parameter must be an array producer. Assuming that the element type of inputArrayProducer is InputType, functionName is the name of a function in the same module with one of the following prototypes:

operator functionName(InputType input, io OutputType output);
operator functionName(InputType input, io OutputType output, Index index);
operator functionName(InputType input, io OutputType output, Index index, Size count)

In cases where the index parameter is present, the index of the element within the array is passed to the generator function; similarly, when the count parameter is present the total number of elements in the array is passed. The result of the createArrayMap call is a value of type ArrayProducer<OutputType>.

An example of an array map:

operator map(String input, io Size output) {
  output = input.length;
}

operator entry() {
  String a[]; a.push("one"); a.push("two"); a.push("three");
  ArrayProducer<String> iap = createConstArray(a);
  report(iap.produce());
  ArrayProducer<Size> oap = createArrayMap(iap, map);
  report(oap.produce());
}

produces:

["one","two","three"]
[3,3,5]

In both cases, you can optionally specify a shared value producer as the last parameter to the create...Map() call. When a shared value producer is provided, the function receives an additional parameter whose type is the value type of the shared value producer. So, for value maps, the prototype of the operator becomes:

operator gen(InputType input, io OutputType output, SharedType sharedValue) {
  ...
}

and for array maps the prototype of the operator becomes:

operator gen(
  in InputType input,
  io OutputType output,
  in Index index,
  in Size count,
  in SharedType sharedValue
  )
{
  ...
}

Note that when using a shared value with an array map you must include the index and count parameters in the operator even if they are unused.

KL Transforms

A value transform in KL is created using the createValueTransform(inputValueProducer, functionName). inputValueProducer must be a value producer, and functionName must be the name of a KL function available in the same KL module that has the operator signature functionName(io ValueType value); the type ValueType must be the same as the value type of the input value producer. The result of the createValueMap call is a value of type ValueProducer<ValueType>. For example, the KL code:

operator transform(io Scalar value) {
  value = sqrt(value);
}

operator entry() {
  ValueProducer<Scalar> vp = createValueTransform(createConstValue(Scalar(3.14)), transform);
  report(vp.produce());
}

produces the output:

1.772004

An array transform in KL is created using the createArrayTransform(inputArrayProducer, functionName). The inputArrayProducer parameter must be an array producer. Assuming that the element type of inputArrayProducer is ElementType, functionName is the name of a function in the same module with one of the following prototypes:

operator functionName(io ElementType element);
operator functionName(io ElementType element, Index index);
operator functionName(io ElementType element, Index index, Size count);

In cases where the index parameter is present, the index of the element within the array is passed to the generator function; similarly, when the count parameter is present the total number of elements in the array is passed. The result of the createArrayTransform call is a value of type ArrayProducer<ElementType>.

An example of an array transform:

operator transform(io Scalar value) {
  value = sqrt(value);
}

operator entry() {
  Scalar ia[]; ia.push(3.14); ia.push(2.71); ia.push(10.0); ia.push(87.32);
  ArrayProducer<Scalar> iap = createConstArray(ia);
  report(iap.produce());
  ArrayProducer<Scalar> oap = createArrayTransform(iap, transform);
  report(oap.produce());
}

produces:

[3.14,2.71,10,87.32]
[1.772004,1.646208,3.162278,9.344517]

In both cases, you can optionally specify a shared value producer as the last parameter to the create...Transform() call. When a shared value producer is provided, the function receives an additional parameter whose type is the value type of the shared value producer. So, for value maps, the prototype of the operator becomes:

operator gen(io ValueType value, SharedType sharedValue) {
  ...
}

and for array maps the prototype of the operator becomes:

operator gen(io ElementType element, Index index, Size count, SharedType sharedValue) {
  ...
}

Note that when using a shared value with an array map you must include the index and count parameters in the operator even if they are unused.

KL Reduce Operations

To create a reduce operation in KL, use the createReduce(inputArrayProducer, functionName) call. inputArrayProducer must be an array producer; assume its element type is InputType. functionName must be the name of a function in the same module with one of the prototypes:

operator functionName(InputType input, io OutputType output);
operator functionName(InputType input, io OutputType output, Index index);
operator functionName(InputType input, io OutputType output, Index index, Size count);

The result of the createReduce call is a value producer with value type OutputType, ie. a result of type ValueProducer<OutputType>. An example of using a reduce operation in KL:

operator generate(io Size value, Index index) {
  value = index + 1;
}

operator reduce(Size input, io Size output) {
  output += input;
}

operator entry() {
  // Report the sum 1+2+...+99+100
  ValueProducer<Size> vp = createReduce(
    createArrayGenerator(
      createConstValue(Size(100)),
      generate
    ),
    reduce
  );
  report(vp.produce());
}

This produces the result:

5050

You can optionally specify a shared value producer as the last parameter to the createReduce() call. When a shared value producer is provided, the function receives an additional parameter whose type is the value type of the shared value producer. The prototype of the operator becomes:

operator reduce(
  in InputType input,
  io OutputType output,
  in Index index,
  in Size count,
  in SharedType sharedValue
  ) {
  ...
}

Note that when using a shared value with an array map you must include the index and count parameters in the operator even if they are unused.

KL Caches

A value cache is created in KL using the createValueCache(inputValueProducer) call. The element type of the resulting value producer is the same as that of inputValueProducer. Example usage of a value cache in KL:

operator gen(io String output) {
  report("  Running generator!");
  output = "Hello";
}

operator entry() {
  // test caching ValueGenerator
  ValueProducer<String> vp1 = createValueCache(createValueGenerator(gen));
  report("vp1 = " + vp1);

  report("Should run generator");
  report("vp1.produce() = " + vp1.produce());

  report("Should not run generator (use cache)");
  report("vp1.produce() = " + vp1.produce());

  vp1.flush();
  report("Should run generator");
  report("vp1.produce() = " + vp1.produce());
}

resulting in:

vp1 = ValueProducer<String>
Should run generator
  Running generator!
vp1.produce() = Hello
Should not run generator (use cache)
vp1.produce() = Hello
Should run generator
  Running generator!
vp1.produce() = Hello

Similarly, an array cache is created using the createArrayProducer(inputArrayProducer) call. The resulting array producer has the same element type as inputArrayProducer. Example usage:

operator generator(
  io Integer value
  )
{
  report("  Running generator");
  value = 42;
}

operator entry() {
  // Generator caching
  ValueProducer<Size> cvg = createConstValue(Size(10));
  ArrayProducer<Integer> gen = createArrayCache(
    createArrayGenerator(cvg, generator)
    );

  report(gen);
  report(" gen.getCount() = " + gen.getCount());

  report("Should run generator 10x");
  for (Index i=0; i<10; ++i)
    report(" gen.produce() = " + gen.produce(i));

  report("Should not run generator (cached)");
  for (Index i=0; i<10; ++i)
    report(" gen.produce() = " + gen.produce(i));

  gen.flush();
  report("Should run generator 10x");
  for (Index i=0; i<10; ++i)
    report(" gen.produce() = " + gen.produce(i));
}

resulting in:

ArrayProducer<Integer>
 gen.getCount() = 10
Should run generator 10x
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
  Running generator
 gen.produce() = 42
Should not run generator (cached)
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
 gen.produce() = 42
Should run generator 10x
  Running generator
 gen.produce() = 42
  Running generator
  Running transform
 tr.produce() = 2
  Running transform
 tr.produce() = 4
  Running transform
 tr.produce() = 6
  Running transform
 tr.produce() = 8
  Running transform
 tr.produce() = 10
  Running transform
 tr.produce() = 12
  Running transform
 tr.produce() = 14
  Running transform
 tr.produce() = 16
  Running transform
 tr.produce() = 18