peon/docs/bytecode.md

# Peon - Bytecode Specification

This document aims to document peon's bytecode as well as how it is (de-)serialized to/from files and
other file-like objects.

## Code Structure

A peon program is compiled into a tightly packed sequence of bytes that contain all the necessary information
the VM needs to execute said program. There is no dependence between the frontend and the backend outside of the
bytecode format (which is implemented in a separate serialiazer module) to allow for maximum modularity.

A peon bytecode dump contains:

- Constants
- The bytecode itself
- Debugging information
- File and version metadata 

## Encoding

### Header

A peon bytecode file starts with the header, which is structured as follows:

- The literal string `PEON_BYTECODE`
- A 3-byte version number (the major, minor and patch versions of the compiler that generated the file as per the SemVer versioning standard)
- The branch name of the repository the compiler was built from, prepended with its length as a 1 byte integer
- The full commit hash (encoded as a 40-byte hex-encoded string) in the aforementioned branch from which the compiler was built from (particularly useful in development builds)
- An 8-byte UNIX timestamp (with Epoch 0 starting at 1/1/1970 12:00 AM) representing the exact date and time of when the file was generated
- A 32-byte, hex-encoded SHA256 hash of the source file's content, used to track file changes

### Line data section

The line data section contains information about each instruction in the code section and associates them
1:1 with a line number in the original source file for easier debugging using run-length encoding. The section's 
size is fixed and is encoded at the beginning as a sequence of 4 bytes (i.e. a single 32 bit integer). The data
in this section can be decoded as explained in [this file](../src/frontend/meta/bytecode.nim#L28), which is quoted
below:
```
[...]
## lines maps bytecode instructions to line numbers using Run
## Length Encoding. Instructions are encoded in groups whose structure
## follows the following schema:
## - The first integer represents the line number
## - The second integer represents the count of whatever comes after it
##  (let's call it c)
## - After c, a sequence of c integers follows
##
## A visual representation may be easier to understand: [1, 2, 3, 4]
## This is to be interpreted as "there are 2 instructions at line 1 whose values
## are 3 and 4"
## This is more efficient than using the naive approach, which would encode
## the same line number multiple times and waste considerable amounts of space.
[...]
```

### Constant section

The constant section contains all the read-only values that the code will need at runtime, such as hardcoded
variable initializers or constant expressions. It is similar to the `.rodata` section of Assembly files, although
the implementation is different. Constants are encoded as a linear sequence of bytes with no type information about
them whatsoever: it is the code that, at runtime, loads each constant (whose type is determined at compile time) onto 
the stack accordingly. For example, a 32 bit integer constant would be encoded as a sequence of 4 bytes, which would
then be loaded by the appropriate `LoadInt32` instruction at runtime. The section's size is fixed and is encoded at
the beginning as a sequence of 4 bytes (i.e. a single 32 bit integer). The constant section may be empty, although in
real-world scenarios it's unlikely that it would.

### Code section

The code section contains the linear sequence of bytecode instructions of a peon program. It is to be read directly 
and without modifications. The section's size is fixed and is encoded at the beginning as a sequence of 3 bytes 
(i.e. a single 24 bit integer).