nocturn9x
/
peon
2
2
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Added mutateOuterScope to noWarn option, improved error formatting, added backend flag, updated help menu

This commit is contained in:
Mattia Giambirtone 2022-12-16 15:11:06 +01:00
parent ffe57c134a
commit cf6d20e757
8 changed files with 120 additions and 92 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
README.md
View File

@ -8,15 +8,15 @@ Peon is a modern, multi-paradigm, async-first programming language with a focus
## Project structure ## Project structure
- `src/` -> Contains the entirety of peon's toolchain - `src/` -> Contains the entirety of peon's toolchain
- `src/frontend/` -> Contains the tokenizer, parser and compiler - `src/frontend/` -> Contains the tokenizer, parser and compiler
- `src/frontend/meta/` -> Contains shared error definitions, AST node and token - `src/frontend/meta/` -> Contains shared error definitions, AST node and token
declarations as well as the bytecode used by the compiler declarations as well as the bytecode used by the compiler
- `src/frontend/compiler` -> Contains the compiler and its various compilation targets - `src/frontend/compiler` -> Contains the compiler and its various compilation targets
- `src/frontend/compiler/targets` -> Contains compilation targets - `src/frontend/compiler/targets` -> Contains compilation targets
- `src/frontend/parsing` -> Contains the tokenizer and parser - `src/frontned/compiler/targets/bytecode` -> Contains the bytecode target
- `src/frontend/compiler/targets/bytecode/util` -> Contains the bytecode debugger/ serializer and other utilities
- `src/frontend/parsing` -> Contains the tokenizer and parser
- `src/backend/` -> Contains the various backends supported by peon (bytecode, native) - `src/backend/` -> Contains the various backends supported by peon (bytecode, native)
- `src/util/` -> Contains utilities such as the bytecode debugger and serializer as well
as procedures to handle multi-byte sequences
- `src/config.nim` -> Contains compile-time configuration variables - `src/config.nim` -> Contains compile-time configuration variables
- `src/main.nim` -> Ties up the whole toolchain together by tokenizing, - `src/main.nim` -> Ties up the whole toolchain together by tokenizing,
parsing, compiling, debugging, (de-)serializing and (optionally) executing peon code parsing, compiling, debugging, (de-)serializing and (optionally) executing peon code
@ -30,19 +30,20 @@ __Note__: For simplicity reasons, the verbs in this section refer to the present
Peon is a multi-paradigm, statically-typed programming language inspired by C, Nim, Python, Rust and C++: it supports modern, high-level Peon is a multi-paradigm, statically-typed programming language inspired by C, Nim, Python, Rust and C++: it supports modern, high-level
features such as automatic type inference, parametrically polymorphic generics, pure functions, closures, interfaces, single inheritance, reference types, templates, raw pointers and exceptions. features such as automatic type inference, parametrically polymorphic generic types, pure functions, closures, interfaces, single inheritance,
reference types, templates, coroutines, raw pointers and exceptions.
The memory management model is rather simple: a Mark and Sweep garbage collector is employed to reclaim unused memory. The memory management model is rather simple: a Mark and Sweep garbage collector is employed to reclaim unused memory.
Peon features a native cooperative concurrency model designed to take advantage of the inherent waiting implied by typical I/O workloads, without the use of more than one OS thread (wherever possible), allowing for much greater efficiency and a smaller memory footprint. The asynchronous model used forces developers to write code that is both easy to reason about, thanks to the [Structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) model that is core to peon's async event loop implementation, and works as expected every time (without dropping signals, exceptions, or task return values). Peon features a native cooperative concurrency model designed to take advantage of the inherent waiting of typical I/O workloads, without the use of more than one OS thread (wherever possible), allowing for much greater efficiency and a smaller memory footprint. The asynchronous model used forces developers to write code that is both easy to reason about, thanks to the [Structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) model that is core to peon's async event loop implementation, and works as expected every time (without dropping signals, exceptions, or task return values).
Other notable features are the ability to define (and overload) custom operators with ease by implementing them as language-level functions, [Universal function call syntax](https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax), [Name stropping](https://en.wikipedia.org/wiki/Stropping_(syntax)) and named scopes. Other notable features are the ability to define (and overload) custom operators with ease by implementing them as language-level functions, [Universal function call syntax](https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax), [Name stropping](https://en.wikipedia.org/wiki/Stropping_(syntax)) and named scopes.
In peon, all objects are first-class: this includes functions, iterators, closures and coroutines. In peon, all objects are first-class (this includes functions, iterators, closures and coroutines).
## Disclaimers ## Disclaimers
**Disclaimer**: The project is still in its very early days: lots of stuff is not implemented, a work in progress or **Disclaimer 1**: The project is still in its very early days: lots of stuff is not implemented, a work in progress or
otherwise outright broken. Feel free to report bugs! otherwise outright broken. Feel free to report bugs!

30
src/config.nim
View File

@ -53,18 +53,24 @@ $ peon file.pbc Run the given Peon bytecode file
Options Options
------- -------
-h, --help Show this help text and exits -h, --help Show this help text and exits
-v, --version Print the current peon version and exits -v, --version Print the current peon version and exits
-s, --string Execute the passed string as if it was a file -s, --string Execute the passed string as if it was a file
-n, --noDump Don't dump the result of compilation to a *.pbc file -n, --noDump Don't dump the result of compilation to a file
-b, --breakpoints Run the debugger at specific bytecode offsets (comma-separated). -b, --breakpoints Run the debugger at specific bytecode offsets (comma-separated).
Only available when compiled with VM debugging on Only available with --backend:bytecode and when compiled with VM
-d, --disassemble Disassemble the given bytecode file instead of executing it debugging on
-m, --mode Set the compilation mode. Acceptable values are 'debug' and -d, --disassemble Disassemble the given bytecode file instead of executing it
'release' (only makes sense with --backend:bytecode)
-c, --compile Compile the code, but do not execute it -m, --mode Set the compilation mode. Acceptable values are 'debug' and
--warnings Turn warnings on/off (default: on). Acceptable values are 'release'
yes/on and no/off -c, --compile Compile the code, but do not execute it
--warnings Turn warnings on/off (default: on). Acceptable values are
yes/on and no/off
--noWarn Disable a specific warning (for example, --noWarn unusedVariable) --noWarn Disable a specific warning (for example, --noWarn unusedVariable)
--showMismatches Show all mismatches when dispatching function calls fails (quite verbose!) --showMismatches Show all mismatches when dispatching function calls fails (quite verbose!)
--backend Select the compilation backend (valid values are: 'c', 'cpp' and 'bytecode'). Note
that the REPL always uses the bytecode target. Defaults to 'bytecode'
-o, --output Rename the output file with this value (with --backend:bytecode, the .pbc extension
is added)
""" """

9
src/frontend/compiler/compiler.nim
View File

@ -50,6 +50,9 @@ export ast, token, symbols, config, errors
type type
PeonBackend* = enum
## An enumeration of the peon backends
Bytecode, NativeC, NativeCpp
PragmaKind* = enum PragmaKind* = enum
## An enumeration of pragma types ## An enumeration of pragma types
Immediate, Immediate,
@ -94,8 +97,6 @@ type
types*: seq[tuple[match: bool, kind: Type]] types*: seq[tuple[match: bool, kind: Type]]
else: else:
discard discard
type
NameKind* {.pure.} = enum NameKind* {.pure.} = enum
## A name enumeration type ## A name enumeration type
None, Module, Argument, Var, Function, CustomType, Enum None, Module, Argument, Var, Function, CustomType, Enum
@ -161,6 +162,7 @@ type
CompileError* = ref object of PeonException CompileError* = ref object of PeonException
node*: ASTNode node*: ASTNode
function*: Declaration function*: Declaration
compiler*: Compiler
Compiler* = ref object of RootObj Compiler* = ref object of RootObj
## A wrapper around the Peon compiler's state ## A wrapper around the Peon compiler's state
@ -212,6 +214,7 @@ type
## Public getters for nicer error formatting ## Public getters for nicer error formatting
proc getCurrentNode*(self: Compiler): ASTNode = (if self.current >= self.ast.len(): self.ast[^1] else: self.ast[self.current - 1]) proc getCurrentNode*(self: Compiler): ASTNode = (if self.current >= self.ast.len(): self.ast[^1] else: self.ast[self.current - 1])
proc getCurrentFunction*(self: Compiler): Declaration {.inline.} = (if self.currentFunction.isNil(): nil else: self.currentFunction.valueType.fun) proc getCurrentFunction*(self: Compiler): Declaration {.inline.} = (if self.currentFunction.isNil(): nil else: self.currentFunction.valueType.fun)
proc getSource*(self: Compiler): string {.inline.} = self.source
## Some forward declarations (some of them arere actually stubs because nim forces forward declarations to be ## Some forward declarations (some of them arere actually stubs because nim forces forward declarations to be
## implemented in the same module). They are methods because we need to dispatch to their actual specific ## implemented in the same module). They are methods because we need to dispatch to their actual specific
@ -266,7 +269,7 @@ proc done*(self: Compiler): bool {.inline.} =
proc error*(self: Compiler, message: string, node: ASTNode = nil) {.inline.} = proc error*(self: Compiler, message: string, node: ASTNode = nil) {.inline.} =
## Raises a CompileError exception ## Raises a CompileError exception
let node = if node.isNil(): self.getCurrentNode() else: node let node = if node.isNil(): self.getCurrentNode() else: node
raise CompileError(msg: message, node: node, line: node.token.line, file: node.file) raise CompileError(msg: message, node: node, line: node.token.line, file: node.file, compiler: self)
proc warning*(self: Compiler, kind: WarningKind, message: string, name: Name = nil, node: ASTNode = nil) = proc warning*(self: Compiler, kind: WarningKind, message: string, name: Name = nil, node: ASTNode = nil) =

0
src/frontend/compiler/targets/native/target.nim → src/frontend/compiler/targets/nativeC/target.nim
View File

0
src/frontend/compiler/targets/native/util/generators.nim → src/frontend/compiler/targets/nativeC/util/generators.nim
View File

4
src/frontend/parsing/parser.nim
View File

@ -213,7 +213,9 @@ proc step(self: Parser, n: int = 1): Token =
proc error(self: Parser, message: string, token: Token = nil) {.raises: [ParseError].} = proc error(self: Parser, message: string, token: Token = nil) {.raises: [ParseError].} =
## Raises a ParseError exception ## Raises a ParseError exception
let token = if token.isNil(): self.getCurrentToken() else: token var token = if token.isNil(): self.getCurrentToken() else: token
if token.kind == EndOfFile:
token = self.peek(-1)
raise ParseError(msg: message, token: token, line: token.line, file: self.file, parser: self) raise ParseError(msg: message, token: token, line: token.line, file: self.file, parser: self)

142
src/main.nim
View File

@ -31,8 +31,7 @@ import std/strformat
import std/strutils import std/strutils
import std/terminal import std/terminal
import std/parseopt import std/parseopt
when debugSerializer: import std/times
import std/times
import std/os import std/os
# Thanks art <3 # Thanks art <3
@ -44,17 +43,22 @@ import jale/keycodes
import jale/multiline import jale/multiline
# Forward declarations proc getLineEditor: LineEditor =
proc getLineEditor: LineEditor result = newLineEditor()
result.prompt = "=> "
result.populateDefaults()
let history = result.plugHistory()
result.bindHistory(history)
proc repl = proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug) =
styledEcho fgMagenta, "Welcome into the peon REPL!" styledEcho fgMagenta, "Welcome into the peon REPL!"
var var
keep = true keep = true
tokens: seq[Token] = @[] tokens: seq[Token] = @[]
tree: seq[Declaration] = @[] tree: seq[Declaration] = @[]
compiled: Chunk = newChunk() compiler = newBytecodeCompiler()
compiled: Chunk
serialized: Serialized serialized: Serialized
tokenizer = newLexer() tokenizer = newLexer()
vm = newPeonVM() vm = newPeonVM()
@ -99,7 +103,7 @@ proc repl =
break break
styledEcho fgGreen, "\t", $token styledEcho fgGreen, "\t", $token
echo "" echo ""
tree = newParser().parse(tokens, "stdin", tokenizer.getLines(), current) tree = newParser().parse(tokens, "stdin", tokenizer.getLines(), current & input & "\n")
if tree.len() == 0: if tree.len() == 0:
continue continue
when debugParser: when debugParser:
@ -107,7 +111,7 @@ proc repl =
for node in tree: for node in tree:
styledEcho fgGreen, "\t", $node styledEcho fgGreen, "\t", $node
echo "" echo ""
compiled = newBytecodeCompiler(replMode=true).compile(tree, "stdin", tokenizer.getLines(), current) compiled = compiler.compile(tree, "stdin", tokenizer.getLines(), current & input & "\n", showMismatches=mismatches, disabledWarnings=warnings, mode=mode)
when debugCompiler: when debugCompiler:
styledEcho fgCyan, "Compilation step:\n" styledEcho fgCyan, "Compilation step:\n"
debugger.disassembleChunk(compiled, "stdin") debugger.disassembleChunk(compiled, "stdin")
@ -155,7 +159,8 @@ proc repl =
proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints: seq[uint64] = @[], dis: bool = false, proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints: seq[uint64] = @[], dis: bool = false,
warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug, run: bool = true) = warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug, run: bool = true,
backend: PeonBackend = PeonBackend.Bytecode) =
var var
tokens: seq[Token] = @[] tokens: seq[Token] = @[]
tree: seq[Declaration] = @[] tree: seq[Declaration] = @[]
@ -174,7 +179,7 @@ proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints
if not fromString: if not fromString:
if not f.endsWith(".pn") and not f.endsWith(".pbc"): if not f.endsWith(".pn") and not f.endsWith(".pbc"):
f &= ".pn" f &= ".pn"
if dis: if backend == PeonBackend.Bytecode and dis:
debugger.disassembleChunk(serializer.loadFile(f).chunk, f) debugger.disassembleChunk(serializer.loadFile(f).chunk, f)
return return
input = readFile(f) input = readFile(f)
@ -201,48 +206,57 @@ proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints
for node in tree: for node in tree:
styledEcho fgGreen, "\t", $node styledEcho fgGreen, "\t", $node
echo "" echo ""
compiled = compiler.compile(tree, f, tokenizer.getLines(), input, disabledWarnings=warnings, showMismatches=mismatches, mode=mode) case backend:
when debugCompiler: of PeonBackend.Bytecode:
styledEcho fgCyan, "Compilation step:\n" compiled = compiler.compile(tree, f, tokenizer.getLines(), input, disabledWarnings=warnings, showMismatches=mismatches, mode=mode)
debugger.disassembleChunk(compiled, f) when debugCompiler:
echo "" styledEcho fgCyan, "Compilation step:\n"
var path = splitFile(f).dir debugger.disassembleChunk(compiled, f)
if path.len() > 0: echo ""
path &= "/" var path = splitFile(f).dir
path &= splitFile(f).name & ".pbc" if path.len() > 0:
if dump and not fromString: path &= "/"
serializer.dumpFile(compiled, f, path) path &= splitFile(f).name & ".pbc"
serialized = serializer.loadFile(path) if dump and not fromString:
else: serializer.dumpFile(compiled, f, path)
serialized = serializer.loadBytes(serializer.dumpBytes(compiled, f)) serialized = serializer.loadFile(path)
else: else:
serialized = serializer.loadBytes(serializer.dumpBytes(compiled, f))
else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "the selected backend is not implemented yet")
elif backend == PeonBackend.Bytecode:
serialized = serializer.loadFile(f) serialized = serializer.loadFile(f)
when debugSerializer: if backend == PeonBackend.Bytecode:
styledEcho fgCyan, "Serialization step: " when debugSerializer:
styledEcho fgBlue, "\t- Peon version: ", fgYellow, &"{serialized.version.major}.{serialized.version.minor}.{serialized.version.patch}", fgBlue, " (commit ", fgYellow, serialized.commit[0..8], fgBlue, ") on branch ", fgYellow, serialized.branch styledEcho fgCyan, "Serialization step: "
stdout.styledWriteLine(fgBlue, "\t- Compilation date & time: ", fgYellow, fromUnix(serialized.compileDate).format("d/M/yyyy HH:mm:ss")) styledEcho fgBlue, "\t- Peon version: ", fgYellow, &"{serialized.version.major}.{serialized.version.minor}.{serialized.version.patch}", fgBlue, " (commit ", fgYellow, serialized.commit[0..8], fgBlue, ") on branch ", fgYellow, serialized.branch
stdout.styledWrite(fgBlue, &"\t- Constants segment: ") stdout.styledWriteLine(fgBlue, "\t- Compilation date & time: ", fgYellow, fromUnix(serialized.compileDate).format("d/M/yyyy HH:mm:ss"))
if serialized.chunk.consts == compiled.consts: stdout.styledWrite(fgBlue, &"\t- Constants segment: ")
styledEcho fgGreen, "OK" if serialized.chunk.consts == compiled.consts:
else: styledEcho fgGreen, "OK"
styledEcho fgRed, "Corrupted" else:
stdout.styledWrite(fgBlue, &"\t- Code segment: ") styledEcho fgRed, "Corrupted"
if serialized.chunk.code == compiled.code: stdout.styledWrite(fgBlue, &"\t- Code segment: ")
styledEcho fgGreen, "OK" if serialized.chunk.code == compiled.code:
else: styledEcho fgGreen, "OK"
styledEcho fgRed, "Corrupted" else:
stdout.styledWrite(fgBlue, "\t- Line info segment: ") styledEcho fgRed, "Corrupted"
if serialized.chunk.lines == compiled.lines: stdout.styledWrite(fgBlue, "\t- Line info segment: ")
styledEcho fgGreen, "OK" if serialized.chunk.lines == compiled.lines:
else: styledEcho fgGreen, "OK"
styledEcho fgRed, "Corrupted" else:
stdout.styledWrite(fgBlue, "\t- CFI segment: ") styledEcho fgRed, "Corrupted"
if serialized.chunk.cfi == compiled.cfi: stdout.styledWrite(fgBlue, "\t- CFI segment: ")
styledEcho fgGreen, "OK" if serialized.chunk.cfi == compiled.cfi:
else: styledEcho fgGreen, "OK"
styledEcho fgRed, "Corrupted" else:
styledEcho fgRed, "Corrupted"
if run: if run:
vm.run(serialized.chunk, breakpoints) case backend:
of PeonBackend.Bytecode:
vm.run(serialized.chunk, breakpoints)
else:
discard
except LexingError: except LexingError:
print(LexingError(getCurrentException())) print(LexingError(getCurrentException()))
except ParseError: except ParseError:
@ -276,6 +290,7 @@ when isMainModule:
var mismatches: bool = false var mismatches: bool = false
var mode: CompileMode = CompileMode.Debug var mode: CompileMode = CompileMode.Debug
var run: bool = true var run: bool = true
var backend: PeonBackend
for kind, key, value in optParser.getopt(): for kind, key, value in optParser.getopt():
case kind: case kind:
of cmdArgument: of cmdArgument:
@ -318,10 +333,12 @@ when isMainModule:
warnings.add(WarningKind.UnusedName) warnings.add(WarningKind.UnusedName)
of "unreachableCode": of "unreachableCode":
warnings.add(WarningKind.UnreachableCode) warnings.add(WarningKind.UnreachableCode)
of "shadowingOuterScope": of "shadowOuterScope":
warnings.add(WarningKind.ShadowOuterScope) warnings.add(WarningKind.ShadowOuterScope)
of "mutateOuterScope":
warnings.add(WarningKind.MutateOuterScope)
else: else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "invalid warning name for option 'warnings'") stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "invalid warning name for option 'noWarn'")
quit() quit()
of "breakpoints": of "breakpoints":
when not debugVM: when not debugVM:
@ -337,6 +354,14 @@ when isMainModule:
dis = true dis = true
of "compile": of "compile":
run = false run = false
of "backend":
case value:
of "bytecode":
backend = PeonBackend.Bytecode
of "c":
backend = PeonBackend.NativeC
of "cpp":
backend = PeonBackend.NativeCpp
else: else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, &"error: unkown option '{key}'") stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, &"error: unkown option '{key}'")
quit() quit()
@ -374,15 +399,6 @@ when isMainModule:
echo "usage: peon [options] [filename.pn]" echo "usage: peon [options] [filename.pn]"
quit() quit()
if file == "": if file == "":
repl() repl(warnings, mismatches, mode)
else: else:
runFile(file, fromString, dump, breaks, dis, warnings, mismatches, mode, run) runFile(file, fromString, dump, breaks, dis, warnings, mismatches, mode, run, backend)
proc getLineEditor: LineEditor =
result = newLineEditor()
result.prompt = "=> "
result.populateDefaults()
let history = result.plugHistory()
result.bindHistory(history)

2
src/util/fmterr.nim
View File

@ -47,7 +47,7 @@ proc print*(exc: CompileError) =
file = relativePath(exc.file, getCurrentDir()) file = relativePath(exc.file, getCurrentDir())
contents = readFile(file).splitLines()[exc.line - 1].strip(chars={'\n'}) contents = readFile(file).splitLines()[exc.line - 1].strip(chars={'\n'})
else: else:
contents = "" contents = exc.compiler.getSource().splitLines()[exc.line - 1].strip(chars={'\n'})
printError(file, contents, exc.line, exc.node.getRelativeBoundaries(), exc.function, printError(file, contents, exc.line, exc.node.getRelativeBoundaries(), exc.function,
exc.msg) exc.msg)