peon/docs/manual.md

# Peon - Manual

Peon is a statically typed, garbage-collected, C-like programming language with
a focus on speed and correctness, but whose main feature is the ability to natively
perform highly efficient parallel I/O operations by implementing the [structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/)
paradigm.

__Note__: Peon is currently a WIP (Work In Progress), and much of the content of this manual is purely theoretical as
of now. If you want to help make this into a reality, feel free to contribute!


## Table of contents

- [Manual](#peon---manual)
- [Design Goals](#design-goals)
- [Examples](#peon-by-example)
- [Grammar](grammar.md)
- [Bytecode](bytecode.md)

## Design Goals 

While peon is inspired from Bob Nystrom's [book](https://craftinginterpreters.com), where he describes a simple toy language
named Lox, the aspiration for it is to become a programming language that could actually be used in the real world. For that
to happen, we need:

- Exceptions (`try/except/finally`)
- An import system (with namespaces, like Python)
- Multithreading support (with a global VM lock when GC'ing)
- Built-in collections (list, tuple, set, etc.)
- Coroutines (w/ structured concurrency)
- Generators
- Generics
- C/Nim FFI
- A C backend (for native speed)
- A package manager

Peon ~~steals~~ borrows many ideas from Python, Nim (the the language peon itself is written in), C and many others.

## Peon by Example

Here follow a few examples of peon code to make it clear what the end product should look like. Note that
not all examples represent working functionality and some of these examples might not be up to date either.
For somewhat more updated code snippets, check the [tests](../tests/) directory.

### Variable declarations

```
var x = 5;        # Inferred type is int64
var y = 3'u16;    # Type is specified as uint16
x = 6;            # Works: type matches
x = 3.0;          # Error: Cannot assign float64 to x
var x = 3.14;     # Error: cannot re-declare x
const z = 6.28;   # Constant declaration
let a = "hi!";    # Cannot be reassigned/mutated
var b: int32 = 5; # Explicit type declaration (TODO)
```

__Note__: Peon supports [name stropping](https://en.wikipedia.org/wiki/Stropping_(syntax)), meaning
    that almost any ASCII sequence of characters can be used as an identifier, including language
    keywords, but stropped names need to be enclosed by matching pairs of backticks (`\``)

### Comments

```
# This is a single-line comment
# Peon has no specific syntax for multi-line comments.

fn id[T: any](x: T): T {
    ## Documentation comments start
    ## with two dashes. They are currently
    ## unused, but will be semantically
    ## relevant in the future. They can
    ## be used to document types, modules
    ## and functions
    return x;
}
```

### Functions

```
fn fib(n: int): int {
    if n < 3 {
        return n;
    }
    return fib(n - 1) +  fib(n - 2);
}

fib(30);
```

### Type declarations (TODO)

```
type Foo = object { # Can also be "ref object" for reference types (managed automatically)
    fieldOne*: int  # Asterisk means the field is public outside the current module
    fieldTwo*: int 
}
```

### Enumeration types (TODO)

```
type SomeEnum = enum {  # Can be mapped to an integer
    KindOne,
    KindTwo
}
```

### Operator overloading

```
operator `+`(a, b: Foo): Foo {
    return Foo(fieldOne: a.fieldOne + b.fieldOne, fieldTwo: a.fieldTwo + b.fieldTwo);
}

Foo(fieldOne: 1, fieldTwo: 3) +  Foo(fieldOne: 2, fieldTwo: 3);  # Foo(fieldOne: 3, fieldTwo: 6)
```

__Note__: Custom operators (e.g. `foo`) can also be defined. The backticks around the plus sign serve to mark it
as an identifier instead of a symbol (which is a requirement for function names, since operators are basically
functions in peon). In fact, even the built-in peon operators are implemented partially in peon (actually, just
their stubs are) and they are then specialized in the compiler to get rid of unnecessary function call overhead.

### Function calls

```
foo(1, 2 + 3, 3.14, bar(baz));
```

__Note__: Operators can be called as functions; If their name is a symbol, just wrap it in backticks like so: 
```
`+`(1, 2)  # Identical to 1 + 2
```

__Note__: Code the likes of `a.b()` is (actually, will be) desugared to `b(a)` if there exists a function 
    `b` whose signature is compatible with the value of `a` (assuming `a` doesn't have a field named `b`,
    in which case the attribute resolution takes precedence)


### Generics

```
fn genericSum[T: Number](a, b: T): T {  # Note: "a, b: T" means that both a and b are of type T
    return a + b;
}

# This allows for a single implementation to be
# re-used multiple times without any code duplication
genericSum(1, 2);
genericSum(3.14, 0.1);
genericSum(1'u8, 250'u8);
```

__Note__: Peon generics are implemented according to a paradigm called [parametric polymorphism](https://en.wikipedia.org/wiki/Parametric_polymorphism). In constrast to the model employed by other languages such as C++, called [ad hoc polymorphism](https://en.wikipedia.org/wiki/Ad_hoc_polymorphism),
where each time a generic function is called with a new type signature it is instantiated and
typechecked (and then compiled), peon checks generics at declaration time and only once: this 
not only saves precious compilation time, but it also allows the compiler to generate a single
implementation for the function (although this is not a requirement) and catches type errors right 
when they occur even if the function is never called, rather than having to wait for the function 
to be called and specialized. Unfortunately, this means that some of the things that are possible 
in, say, C++ templates are just not possible with peon generics. As an example, take this code snippet:

```
fn add[T: any](a, b: T): T {
    return a + b;
}
```

While the intent of this code is clear and makes sense semantically speaking, peon will refuse
to compile it because it cannot prove that the `+` operator is defined on every type (in fact, 
it's only defined for numbers): this is a feature. If peon allowed it, `any` could be used to
escape the safety of the type system (for example, calling `add` with `string`s, which may or
may not be what you want).

Since the goal for peon is to not constrain the developer into one specific programming paradigm,
it also implements a secondary, different, generic mechanism using the `auto` type. The above code
could be rewritten to work as follows:

```
fn add(a, b: auto): auto {
    return a + b;
}
```

When using automatic types, peon will behave similarly to C++ (think: templates) and only specialize,
typecheck and compile the function once it is called with a given type signature. For this reason,
automatic and parametrically polymorphic types cannot be used together in peon code.

Another noteworthy concept to keep in mind is that of type unions. For example, take this snippet:

```
fn foo(x: int32): int32 {
    return x;
}


fn foo(x: int): int {
    return x;
}


fn identity[T: int | int32](x: T): T {
    return foo(x);
}
```

This code will, again, fail to compile: this is because as far as peon is concerned, `foo` is not
defined for both `int` and `int32` _at the same time_. In order for that to work, `foo` would need
to be rewritten with `T: int32 | int` as its generic argument type in order to avoid the ambiguity
(or `identity` could be rewritten to use automatic types instead, both are viable options). Obviously,
the above snippet would fail to compile if `foo` were not defined for all the types specified in the
type constraint for `identity` as well (this is because, counterintuitively, matching a generic constraint
such as `int32 | int` does _not_ mean "either of these types", but rather "_both_ of these types at 
once").


#### More generics

```
fn genericSth[T: someTyp, K: someTyp2](a: T, b: K) {  # Note: no return type == void function
    # code...
}

genericSth(1, 3.0);
```


#### Even more generics

```
type Box*[T: Number] = object {
    num: T;
}

var boxFloat = Box[float](1.0);
var boxInt = Box[int](1);
```

__Note__: The `*` modifier to make a name visible outside the current module must be put
__before__ the generic constraints, so only `fn foo*[T](a: T) {}` is the correct syntax.



### Forward declarations

```
fn someF: int;  # Semicolon and no body == forward declaration

print(someF());  # Prints 42

fn someF: int {
    return 42;
}
```

__Note__: A function that is forward-declared __must__ be implemented in the same module as 
the forward declaration.

### Generators

```
generator count(n: int): int {
    while n > 0 {
        yield n;
        n -= 1;
    }
}

foreach n in count(10) {
    print(n);
}
```


### Coroutines

```
import concur;
import http;


coroutine req(url: string): string {
    return (await http.AsyncClient().get(url)).content;
}


coroutine main(urls: list[string]) {
    pool = concur.pool();  # Creates a task pool: like a nursery in njsmith's article
    foreach url in urls {
        pool.spawn(req, urls);
    }
    # The pool has internal machinery that makes the parent
    # task wait until all child exit! When this function
    # returns, ALL child tasks will have exited somehow.
    # Exceptions and return values propagate neatly, too.
}


concur.run(main, newList[string]("https://google.com", "https://debian.org"))
```