peon/docs/manual.md

Peon - Manual

Peon is a statically typed, garbage-collected, C-like programming language with a focus on speed and correctness, but whose main feature is the ability to natively perform highly efficient parallel I/O operations by implementing the structured concurrency paradigm.

Note: Peon is currently a WIP (Work In Progress), and much of the content of this manual is purely theoretical as of now. If you want to help make this into a reality, feel free to contribute!

Table of contents

• Manual
• Design Goals
• Examples
• Grammar
• Bytecode

Design Goals

While peon is inspired from Bob Nystrom's book, where he describes a simple toy language named Lox, the aspiration for it is to become a programming language that could actually be used in the real world. For that to happen, we need:

• Exceptions (try/except/finally)
• An import system (with namespaces, like Python)
• Multithreading support (with a global VM lock when GC'ing)
• Built-in collections (list, tuple, set, etc.)
• Coroutines (w/ structured concurrency)
• Generators
• Generics
• C/Nim FFI
• A C backend (for native speed)
• A package manager

Peon steals borrows many ideas from Python, Nim (the the language peon itself is written in), C and many others.

Peon by Example

Here follow a few examples of peon code to make it clear what the end product should look like. Note that not all examples represent working functionality and some of these examples might not be up to date either. For somewhat more updated code snippets, check the tests directory.

Variable declarations

var x = 5;        # Inferred type is int64
var y = 3'u16;    # Type is specified as uint16
x = 6;            # Works: type matches
x = 3.0;          # Error: Cannot assign float64 to x
var x = 3.14;     # Error: cannot re-declare x
const z = 6.28;   # Constant declaration
let a = "hi!";    # Cannot be reassigned/mutated
var b: int32 = 5; # Explicit type declaration (TODO)

Note: Peon supports name stropping, meaning that almost any ASCII sequence of characters can be used as an identifier, including language keywords, but stropped names need to be enclosed by matching pairs of backticks (```)

Comments

# This is a single-line comment
# Peon has no specific syntax for multi-line comments.

fn id[T: any](x: T): T {
    ## Documentation comments start
    ## with two dashes. They are currently
    ## unused, but will be semantically
    ## relevant in the future. They can
    ## be used to document types, modules
    ## and functions
    return x;
}

Functions

fn fib(n: int): int {
    if n < 3 {
        return n;
    }
    return fib(n - 1) +  fib(n - 2);
}

fib(30);

Type declarations (TODO)

type Foo = object { # Can also be "ref object" for reference types (managed automatically)
    fieldOne*: int  # Asterisk means the field is public outside the current module
    fieldTwo*: int 
}

Enumeration types (TODO)

type SomeEnum = enum {  # Can be mapped to an integer
    KindOne,
    KindTwo
}

Operator overloading

operator `+`(a, b: Foo): Foo {
    return Foo(fieldOne: a.fieldOne + b.fieldOne, fieldTwo: a.fieldTwo + b.fieldTwo);
}

Foo(fieldOne: 1, fieldTwo: 3) +  Foo(fieldOne: 2, fieldTwo: 3);  # Foo(fieldOne: 3, fieldTwo: 6)

Note: Custom operators (e.g. foo) can also be defined. The backticks around the plus sign serve to mark it as an identifier instead of a symbol (which is a requirement for function names, since operators are basically functions in peon). In fact, even the built-in peon operators are implemented partially in peon (actually, just their stubs are) and they are then specialized in the compiler to get rid of unnecessary function call overhead.

Function calls

foo(1, 2 + 3, 3.14, bar(baz));

Note: Operators can be called as functions; If their name is a symbol, just wrap it in backticks like so:

`+`(1, 2)  # Identical to 1 + 2

Note: Code the likes of a.b() is (actually, will be) desugared to b(a) if there exists a function b whose signature is compatible with the value of a (assuming a doesn't have a field named b, in which case the attribute resolution takes precedence)

Generics

fn genericSum[T: Number](a, b: T): T {  # Note: "a, b: T" means that both a and b are of type T
    return a + b;
}

# This allows for a single implementation to be
# re-used multiple times without any code duplication
genericSum(1, 2);
genericSum(3.14, 0.1);
genericSum(1'u8, 250'u8);

Note: Peon generics are implemented according to a paradigm called parametric polymorphism. In constrast to the model employed by other languages such as C++, called ad hoc polymorphism, where each time a generic function is called with a new type signature it is instantiated and typechecked (and then compiled), peon checks generics at declaration time and only once: this not only saves precious compilation time, but it also allows the compiler to generate a single implementation for the function (although this is not a requirement) and catches type errors right when they occur even if the function is never called, rather than having to wait for the function to be called and specialized. Unfortunately, this means that some of the things that are possible in, say, C++ templates are just not possible with peon generics. As an example, take this code snippet:

fn add[T: any](a, b: T): T {
    return a + b;
}

While the intent of this code is clear and makes sense semantically speaking, peon will refuse to compile it because it cannot prove that the + operator is defined on every type (in fact, it's only defined for numbers): this is a feature. If peon allowed it, any could be used to escape the safety of the type system (for example, calling add with strings, which may or may not be what you want).

Since the goal for peon is to not constrain the developer into one specific programming paradigm, it also implements a secondary, different, generic mechanism using the auto type. The above code could be rewritten to work as follows:

fn add(a, b: auto): auto {
    return a + b;
}

When using automatic types, peon will behave similarly to C++ (think: templates) and only specialize, typecheck and compile the function once it is called with a given type signature. For this reason, automatic and parametrically polymorphic types cannot be used together in peon code.

Another noteworthy concept to keep in mind is that of type unions. For example, take this snippet:

fn foo(x: int32): int32 {
    return x;
}


fn foo(x: int): int {
    return x;
}


fn identity[T: int | int32](x: T): T {
    return foo(x);
}

This code will, again, fail to compile: this is because as far as peon is concerned, foo is not defined for both int and int32 at the same time. In order for that to work, foo would need to be rewritten with T: int32 | int as its generic argument type in order to avoid the ambiguity (or identity could be rewritten to use automatic types instead, both are viable options). Obviously, the above snippet would fail to compile if foo were not defined for all the types specified in the type constraint for identity as well (this is because, counterintuitively, matching a generic constraint such as int32 | int does not mean "either of these types", but rather "both of these types at once").

More generics

fn genericSth[T: someTyp, K: someTyp2](a: T, b: K) {  # Note: no return type == void function
    # code...
}

genericSth(1, 3.0);

Even more generics

type Box*[T: Number] = object {
    num: T;
}

var boxFloat = Box[float](1.0);
var boxInt = Box[int](1);

Note: The * modifier to make a name visible outside the current module must be put before the generic constraints, so only fn foo*[T](a: T) {} is the correct syntax.

Forward declarations

fn someF: int;  # Semicolon and no body == forward declaration

print(someF());  # Prints 42

fn someF: int {
    return 42;
}

Note: A function that is forward-declared must be implemented in the same module as the forward declaration.

Generators

generator count(n: int): int {
    while n > 0 {
        yield n;
        n -= 1;
    }
}

foreach n in count(10) {
    print(n);
}

Coroutines

import concur;
import http;


coroutine req(url: string): string {
    return (await http.AsyncClient().get(url)).content;
}


coroutine main(urls: list[string]) {
    pool = concur.pool();  # Creates a task pool: like a nursery in njsmith's article
    foreach url in urls {
        pool.spawn(req, urls);
    }
    # The pool has internal machinery that makes the parent
    # task wait until all child exit! When this function
    # returns, ALL child tasks will have exited somehow.
    # Exceptions and return values propagate neatly, too.
}


concur.run(main, newList[string]("https://google.com", "https://debian.org"))