The peon programming language

Peon is a simple, functional, async-first programming language with a focus on correctness and speed.

Go to the Manual

Project structure

• src/ -> Contains the entirety of peon's toolchain
  • src/memory/ -> Contains peon's memory allocator and GC GC is in the VM right now, sorry!
  • src/frontend/ -> Contains the tokenizer, parser and compiler
    • src/frontend/meta/ -> Contains shared error definitions, AST node and token declarations as well as the bytecode used by the compiler
  • src/backend/ -> Contains the peon VM and type system
  • src/util/ -> Contains utilities such as the bytecode debugger and serializer as well as procedures to handle multi-byte sequences
  • src/config.nim -> Contains compile-time configuration variables
  • src/main.nim -> Ties up the whole toolchain together by tokenizing, parsing, compiling, debugging, (de-)serializing and executing peon code
• docs/ -> Contains documentation for various components of peon (bytecode, syntax, etc.)
• tests/ -> Contains tests (both in peon and Nim) for the toolchain

Credits

• Araq, for creating the amazing language that is Nim (as well as all of its contributors!)
• Guido Van Rossum, aka the chad who created Python and its awesome community and resources
• The Nim community and contributors, for making Nim what it is today
• Bob Nystrom, for his amazing book that inspired me and taught me how to actually make a programming language (kinda, I'm still very dumb)
• Njsmith, for his awesome articles on structured concurrency
• All the amazing people in the r/ProgrammingLanguages subreddit and its Discord server
• Art <3
• Everyone to listened (and still listens to) me ramble about compilers, programming languages and the likes (and for giving me ideas and testing peon!)
• ... More? (I'd thank the contributors but it's just me :P)
• Me! I guess

Project State

Disclaimer: The project is still in its very early days: lots of stuff is not implemented, a work in progress or otherwise outright broken. Feel free to report bugs!

Disclaimer 2: Currently the REPL is very basic (it incrementally joins your lines with a newline, as if it was compiling a new file every time), because incremental compilation is designed for modules and it doesn't play well with the interactive nature of a REPL session. To show the current state of the REPL, type #show (this will print all the code that has been typed so far), while to reset everything (will un-import modules and stuff), type #reset. You can also type #clear if you want a clean slate to type in, but note that it won't reset the REPL state.

Also, yes: peon is yet another programming language inspired by Bob's book, but it is also very different from Lox, which is an object-oriented, dynamically typed and very high level programming language, whereas peon is a statically-typed, functional language which aims to allow low-level interfacing with C and Nim code while being a breeze to use.

Also, peon will feature structured concurrency with coroutines (think Futures/Fibers but without callback hell). Since, unlike Lox, peon isn't a toy language, there's obviously plans to implement creature comforts like an import system, exception handling, a package manager, etc.

TODO List

In no particular order, here's a list of stuff that's done/to do (might be incomplete/out of date):

Toolchain:

• Tokenizer (with dynamic symbol table) ✅
• Parser (with support for custom operators, even builtins) ✅
• Compiler ✅
• VM ✅
• Bytecode (de-)serializer ✅
• Static code debugger ✅
• Runtime debugger/inspection tool ✅

Type system:

• Custom types
• Intrinsics ✅
• Generics ✅
• Functions ✅
• Closures ✅

Misc:

• Pragmas ✅
• Attribute resolution
• UFCS
• Import system with namespaces and visibility control ✅

Note: Since peon isn't complete yet, the toolchain (lexer, parser, compiler and VM) may change, but since they currently work and run, they're marked as "complete" even if technically speaking they aren't. They're functional, though

Feature wishlist

Here's a random list of high-level features I would like peon to have and that I think are kinda neat (some may have been implemented alredady):

• Reference types are not nullable by default (must use #pragma[nullable])
• Easy C/Nim interop via FFI
• C/C++ backend
• Nim backend
• Structured concurrency
• Capability-based programming (i.e. functions are passed objects to act on the real world)
• Parametric Polymorphism (with untagged typed unions) ✅
• Simple OOP (with multiple dispatch!)
• RTTI, with methods that dispatch at runtime based on the true type of a value
• Limited compile-time evaluation (embed the Peon VM in the C/C++/Nim backend and use that to execute peon code at compile time)

The name

The name for peon comes from Productive2's genius cute brain, and is a result of shortening the name of the fastest animal on earth: the Peregrine Falcon. I guess I wanted this to mean peon will be blazing fast (I certainly hope so!)

Peon needs you.

No, but really. I need help. This project is huge and (IMHO) awesome, but there's a lot of non-trivial work to do and doing it with other people is just plain more fun and rewarding. If you want to get involved, definitely try contacting me or open an issue/PR!