nocturn9x
/
peon
2
2
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Added mutateOuterScope to noWarn option, improved error formatting, added backend flag, updated help menu

This commit is contained in:
Mattia Giambirtone 2022-12-16 15:11:06 +01:00
parent ffe57c134a
commit cf6d20e757
8 changed files with 120 additions and 92 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
README.md
View File

@ -8,15 +8,15 @@ Peon is a modern, multi-paradigm, async-first programming language with a focus
## Project structure
- `src/` -> Contains the entirety of peon's toolchain
- `src/frontend/` -> Contains the tokenizer, parser and compiler
- `src/frontend/meta/` -> Contains shared error definitions, AST node and token
declarations as well as the bytecode used by the compiler
- `src/frontend/compiler` -> Contains the compiler and its various compilation targets
- `src/frontend/compiler/targets` -> Contains compilation targets
- `src/frontend/parsing` -> Contains the tokenizer and parser
- `src/frontend/` -> Contains the tokenizer, parser and compiler
- `src/frontend/meta/` -> Contains shared error definitions, AST node and token
declarations as well as the bytecode used by the compiler
- `src/frontend/compiler` -> Contains the compiler and its various compilation targets
- `src/frontend/compiler/targets` -> Contains compilation targets
- `src/frontned/compiler/targets/bytecode` -> Contains the bytecode target
- `src/frontend/compiler/targets/bytecode/util` -> Contains the bytecode debugger/ serializer and other utilities
- `src/frontend/parsing` -> Contains the tokenizer and parser
- `src/backend/` -> Contains the various backends supported by peon (bytecode, native)
- `src/util/` -> Contains utilities such as the bytecode debugger and serializer as well
as procedures to handle multi-byte sequences
- `src/config.nim` -> Contains compile-time configuration variables
- `src/main.nim` -> Ties up the whole toolchain together by tokenizing,
parsing, compiling, debugging, (de-)serializing and (optionally) executing peon code
@ -30,19 +30,20 @@ __Note__: For simplicity reasons, the verbs in this section refer to the present
Peon is a multi-paradigm, statically-typed programming language inspired by C, Nim, Python, Rust and C++: it supports modern, high-level
features such as automatic type inference, parametrically polymorphic generics, pure functions, closures, interfaces, single inheritance, reference types, templates, raw pointers and exceptions.
features such as automatic type inference, parametrically polymorphic generic types, pure functions, closures, interfaces, single inheritance,
reference types, templates, coroutines, raw pointers and exceptions.
The memory management model is rather simple: a Mark and Sweep garbage collector is employed to reclaim unused memory.
Peon features a native cooperative concurrency model designed to take advantage of the inherent waiting implied by typical I/O workloads, without the use of more than one OS thread (wherever possible), allowing for much greater efficiency and a smaller memory footprint. The asynchronous model used forces developers to write code that is both easy to reason about, thanks to the [Structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) model that is core to peon's async event loop implementation, and works as expected every time (without dropping signals, exceptions, or task return values).
Peon features a native cooperative concurrency model designed to take advantage of the inherent waiting of typical I/O workloads, without the use of more than one OS thread (wherever possible), allowing for much greater efficiency and a smaller memory footprint. The asynchronous model used forces developers to write code that is both easy to reason about, thanks to the [Structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/) model that is core to peon's async event loop implementation, and works as expected every time (without dropping signals, exceptions, or task return values).
Other notable features are the ability to define (and overload) custom operators with ease by implementing them as language-level functions, [Universal function call syntax](https://en.wikipedia.org/wiki/Uniform_Function_Call_Syntax), [Name stropping](https://en.wikipedia.org/wiki/Stropping_(syntax)) and named scopes.
In peon, all objects are first-class: this includes functions, iterators, closures and coroutines.
In peon, all objects are first-class (this includes functions, iterators, closures and coroutines).
## Disclaimers
**Disclaimer**: The project is still in its very early days: lots of stuff is not implemented, a work in progress or
**Disclaimer 1**: The project is still in its very early days: lots of stuff is not implemented, a work in progress or
otherwise outright broken. Feel free to report bugs!

30
src/config.nim
View File

@ -53,18 +53,24 @@ $ peon file.pbc Run the given Peon bytecode file
Options
-------
-h, --help Show this help text and exits
-v, --version Print the current peon version and exits
-s, --string Execute the passed string as if it was a file
-n, --noDump Don't dump the result of compilation to a *.pbc file
-b, --breakpoints Run the debugger at specific bytecode offsets (comma-separated).
Only available when compiled with VM debugging on
-d, --disassemble Disassemble the given bytecode file instead of executing it
-m, --mode Set the compilation mode. Acceptable values are 'debug' and
'release'
-c, --compile Compile the code, but do not execute it
--warnings Turn warnings on/off (default: on). Acceptable values are
yes/on and no/off
-h, --help Show this help text and exits
-v, --version Print the current peon version and exits
-s, --string Execute the passed string as if it was a file
-n, --noDump Don't dump the result of compilation to a file
-b, --breakpoints Run the debugger at specific bytecode offsets (comma-separated).
Only available with --backend:bytecode and when compiled with VM
debugging on
-d, --disassemble Disassemble the given bytecode file instead of executing it
(only makes sense with --backend:bytecode)
-m, --mode Set the compilation mode. Acceptable values are 'debug' and
'release'
-c, --compile Compile the code, but do not execute it
--warnings Turn warnings on/off (default: on). Acceptable values are
yes/on and no/off
--noWarn Disable a specific warning (for example, --noWarn unusedVariable)
--showMismatches Show all mismatches when dispatching function calls fails (quite verbose!)
--backend Select the compilation backend (valid values are: 'c', 'cpp' and 'bytecode'). Note
that the REPL always uses the bytecode target. Defaults to 'bytecode'
-o, --output Rename the output file with this value (with --backend:bytecode, the .pbc extension
is added)
"""

9
src/frontend/compiler/compiler.nim
View File

@ -50,6 +50,9 @@ export ast, token, symbols, config, errors
type
PeonBackend* = enum
## An enumeration of the peon backends
Bytecode, NativeC, NativeCpp
PragmaKind* = enum
## An enumeration of pragma types
Immediate,
@ -94,8 +97,6 @@ type
types*: seq[tuple[match: bool, kind: Type]]
else:
discard
type
NameKind* {.pure.} = enum
## A name enumeration type
None, Module, Argument, Var, Function, CustomType, Enum
@ -161,6 +162,7 @@ type
CompileError* = ref object of PeonException
node*: ASTNode
function*: Declaration
compiler*: Compiler
Compiler* = ref object of RootObj
## A wrapper around the Peon compiler's state
@ -212,6 +214,7 @@ type
## Public getters for nicer error formatting
proc getCurrentNode*(self: Compiler): ASTNode = (if self.current >= self.ast.len(): self.ast[^1] else: self.ast[self.current - 1])
proc getCurrentFunction*(self: Compiler): Declaration {.inline.} = (if self.currentFunction.isNil(): nil else: self.currentFunction.valueType.fun)
proc getSource*(self: Compiler): string {.inline.} = self.source
## Some forward declarations (some of them arere actually stubs because nim forces forward declarations to be
## implemented in the same module). They are methods because we need to dispatch to their actual specific
@ -266,7 +269,7 @@ proc done*(self: Compiler): bool {.inline.} =
proc error*(self: Compiler, message: string, node: ASTNode = nil) {.inline.} =
## Raises a CompileError exception
let node = if node.isNil(): self.getCurrentNode() else: node
raise CompileError(msg: message, node: node, line: node.token.line, file: node.file)
raise CompileError(msg: message, node: node, line: node.token.line, file: node.file, compiler: self)
proc warning*(self: Compiler, kind: WarningKind, message: string, name: Name = nil, node: ASTNode = nil) =

0
src/frontend/compiler/targets/native/target.nim → src/frontend/compiler/targets/nativeC/target.nim
View File

0
src/frontend/compiler/targets/native/util/generators.nim → src/frontend/compiler/targets/nativeC/util/generators.nim
View File

4
src/frontend/parsing/parser.nim
View File

@ -213,7 +213,9 @@ proc step(self: Parser, n: int = 1): Token =
proc error(self: Parser, message: string, token: Token = nil) {.raises: [ParseError].} =
## Raises a ParseError exception
let token = if token.isNil(): self.getCurrentToken() else: token
var token = if token.isNil(): self.getCurrentToken() else: token
if token.kind == EndOfFile:
token = self.peek(-1)
raise ParseError(msg: message, token: token, line: token.line, file: self.file, parser: self)

142
src/main.nim
View File

@ -31,8 +31,7 @@ import std/strformat
import std/strutils
import std/terminal
import std/parseopt
when debugSerializer:
import std/times
import std/times
import std/os
# Thanks art <3
@ -44,17 +43,22 @@ import jale/keycodes
import jale/multiline
# Forward declarations
proc getLineEditor: LineEditor
proc getLineEditor: LineEditor =
result = newLineEditor()
result.prompt = "=> "
result.populateDefaults()
let history = result.plugHistory()
result.bindHistory(history)
proc repl =
proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug) =
styledEcho fgMagenta, "Welcome into the peon REPL!"
var
keep = true
tokens: seq[Token] = @[]
tree: seq[Declaration] = @[]
compiled: Chunk = newChunk()
compiler = newBytecodeCompiler()
compiled: Chunk
serialized: Serialized
tokenizer = newLexer()
vm = newPeonVM()
@ -99,7 +103,7 @@ proc repl =
break
styledEcho fgGreen, "\t", $token
echo ""
tree = newParser().parse(tokens, "stdin", tokenizer.getLines(), current)
tree = newParser().parse(tokens, "stdin", tokenizer.getLines(), current & input & "\n")
if tree.len() == 0:
continue
when debugParser:
@ -107,7 +111,7 @@ proc repl =
for node in tree:
styledEcho fgGreen, "\t", $node
echo ""
compiled = newBytecodeCompiler(replMode=true).compile(tree, "stdin", tokenizer.getLines(), current)
compiled = compiler.compile(tree, "stdin", tokenizer.getLines(), current & input & "\n", showMismatches=mismatches, disabledWarnings=warnings, mode=mode)
when debugCompiler:
styledEcho fgCyan, "Compilation step:\n"
debugger.disassembleChunk(compiled, "stdin")
@ -155,7 +159,8 @@ proc repl =
proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints: seq[uint64] = @[], dis: bool = false,
warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug, run: bool = true) =
warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: CompileMode = Debug, run: bool = true,
backend: PeonBackend = PeonBackend.Bytecode) =
var
tokens: seq[Token] = @[]
tree: seq[Declaration] = @[]
@ -174,7 +179,7 @@ proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints
if not fromString:
if not f.endsWith(".pn") and not f.endsWith(".pbc"):
f &= ".pn"
if dis:
if backend == PeonBackend.Bytecode and dis:
debugger.disassembleChunk(serializer.loadFile(f).chunk, f)
return
input = readFile(f)
@ -201,48 +206,57 @@ proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints
for node in tree:
styledEcho fgGreen, "\t", $node
echo ""
compiled = compiler.compile(tree, f, tokenizer.getLines(), input, disabledWarnings=warnings, showMismatches=mismatches, mode=mode)
when debugCompiler:
styledEcho fgCyan, "Compilation step:\n"
debugger.disassembleChunk(compiled, f)
echo ""
var path = splitFile(f).dir
if path.len() > 0:
path &= "/"
path &= splitFile(f).name & ".pbc"
if dump and not fromString:
serializer.dumpFile(compiled, f, path)
serialized = serializer.loadFile(path)
else:
serialized = serializer.loadBytes(serializer.dumpBytes(compiled, f))
else:
case backend:
of PeonBackend.Bytecode:
compiled = compiler.compile(tree, f, tokenizer.getLines(), input, disabledWarnings=warnings, showMismatches=mismatches, mode=mode)
when debugCompiler:
styledEcho fgCyan, "Compilation step:\n"
debugger.disassembleChunk(compiled, f)
echo ""
var path = splitFile(f).dir
if path.len() > 0:
path &= "/"
path &= splitFile(f).name & ".pbc"
if dump and not fromString:
serializer.dumpFile(compiled, f, path)
serialized = serializer.loadFile(path)
else:
serialized = serializer.loadBytes(serializer.dumpBytes(compiled, f))
else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "the selected backend is not implemented yet")
elif backend == PeonBackend.Bytecode:
serialized = serializer.loadFile(f)
when debugSerializer:
styledEcho fgCyan, "Serialization step: "
styledEcho fgBlue, "\t- Peon version: ", fgYellow, &"{serialized.version.major}.{serialized.version.minor}.{serialized.version.patch}", fgBlue, " (commit ", fgYellow, serialized.commit[0..8], fgBlue, ") on branch ", fgYellow, serialized.branch
stdout.styledWriteLine(fgBlue, "\t- Compilation date & time: ", fgYellow, fromUnix(serialized.compileDate).format("d/M/yyyy HH:mm:ss"))
stdout.styledWrite(fgBlue, &"\t- Constants segment: ")
if serialized.chunk.consts == compiled.consts:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, &"\t- Code segment: ")
if serialized.chunk.code == compiled.code:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, "\t- Line info segment: ")
if serialized.chunk.lines == compiled.lines:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, "\t- CFI segment: ")
if serialized.chunk.cfi == compiled.cfi:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
if backend == PeonBackend.Bytecode:
when debugSerializer:
styledEcho fgCyan, "Serialization step: "
styledEcho fgBlue, "\t- Peon version: ", fgYellow, &"{serialized.version.major}.{serialized.version.minor}.{serialized.version.patch}", fgBlue, " (commit ", fgYellow, serialized.commit[0..8], fgBlue, ") on branch ", fgYellow, serialized.branch
stdout.styledWriteLine(fgBlue, "\t- Compilation date & time: ", fgYellow, fromUnix(serialized.compileDate).format("d/M/yyyy HH:mm:ss"))
stdout.styledWrite(fgBlue, &"\t- Constants segment: ")
if serialized.chunk.consts == compiled.consts:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, &"\t- Code segment: ")
if serialized.chunk.code == compiled.code:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, "\t- Line info segment: ")
if serialized.chunk.lines == compiled.lines:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
stdout.styledWrite(fgBlue, "\t- CFI segment: ")
if serialized.chunk.cfi == compiled.cfi:
styledEcho fgGreen, "OK"
else:
styledEcho fgRed, "Corrupted"
if run:
vm.run(serialized.chunk, breakpoints)
case backend:
of PeonBackend.Bytecode:
vm.run(serialized.chunk, breakpoints)
else:
discard
except LexingError:
print(LexingError(getCurrentException()))
except ParseError:
@ -276,6 +290,7 @@ when isMainModule:
var mismatches: bool = false
var mode: CompileMode = CompileMode.Debug
var run: bool = true
var backend: PeonBackend
for kind, key, value in optParser.getopt():
case kind:
of cmdArgument:
@ -318,10 +333,12 @@ when isMainModule:
warnings.add(WarningKind.UnusedName)
of "unreachableCode":
warnings.add(WarningKind.UnreachableCode)
of "shadowingOuterScope":
of "shadowOuterScope":
warnings.add(WarningKind.ShadowOuterScope)
of "mutateOuterScope":
warnings.add(WarningKind.MutateOuterScope)
else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "invalid warning name for option 'warnings'")
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, "invalid warning name for option 'noWarn'")
quit()
of "breakpoints":
when not debugVM:
@ -337,6 +354,14 @@ when isMainModule:
dis = true
of "compile":
run = false
of "backend":
case value:
of "bytecode":
backend = PeonBackend.Bytecode
of "c":
backend = PeonBackend.NativeC
of "cpp":
backend = PeonBackend.NativeCpp
else:
stderr.styledWriteLine(fgRed, styleBright, "Error: ", fgDefault, &"error: unkown option '{key}'")
quit()
@ -374,15 +399,6 @@ when isMainModule:
echo "usage: peon [options] [filename.pn]"
quit()
if file == "":
repl()
repl(warnings, mismatches, mode)
else:
runFile(file, fromString, dump, breaks, dis, warnings, mismatches, mode, run)
proc getLineEditor: LineEditor =
result = newLineEditor()
result.prompt = "=> "
result.populateDefaults()
let history = result.plugHistory()
result.bindHistory(history)
runFile(file, fromString, dump, breaks, dis, warnings, mismatches, mode, run, backend)

2
src/util/fmterr.nim
View File

@ -47,7 +47,7 @@ proc print*(exc: CompileError) =
file = relativePath(exc.file, getCurrentDir())
contents = readFile(file).splitLines()[exc.line - 1].strip(chars={'\n'})
else:
contents = ""
contents = exc.compiler.getSource().splitLines()[exc.line - 1].strip(chars={'\n'})
printError(file, contents, exc.line, exc.node.getRelativeBoundaries(), exc.function,
exc.msg)