Minor additions and cleanup
This commit is contained in:
parent
e97653cbc7
commit
ace04ee34c
21
README.md
21
README.md
|
|
@ -5,25 +5,6 @@ Peon is a modern, multi-paradigm, async-first programming language with a focus
|
|||
[Go to the Manual](docs/manual.md)
|
||||
|
||||
|
||||
## Project structure
|
||||
|
||||
- `src/` -> Contains the entirety of peon's toolchain
|
||||
- `src/frontend/` -> Contains the tokenizer, parser and compiler
|
||||
- `src/frontend/meta/` -> Contains shared error definitions, AST node and token
|
||||
declarations as well as the bytecode used by the compiler
|
||||
- `src/frontend/compiler` -> Contains the compiler and its various compilation targets
|
||||
- `src/frontend/compiler/targets` -> Contains compilation targets
|
||||
- `src/frontned/compiler/targets/bytecode` -> Contains the bytecode target
|
||||
- `src/frontend/compiler/targets/bytecode/util` -> Contains the bytecode debugger/ serializer and other utilities
|
||||
- `src/frontend/parsing` -> Contains the tokenizer and parser
|
||||
- `src/backend/` -> Contains the various backends supported by peon (bytecode, native)
|
||||
- `src/config.nim` -> Contains compile-time configuration variables
|
||||
- `src/main.nim` -> Ties up the whole toolchain together by tokenizing,
|
||||
parsing, compiling, debugging, (de-)serializing and (optionally) executing peon code
|
||||
- `docs/` -> Contains documentation for various components of peon (bytecode, syntax, etc.)
|
||||
- `tests/` -> Contains peon tests for the toolchain
|
||||
|
||||
|
||||
## What's peon?
|
||||
|
||||
__Note__: For simplicity reasons, the verbs in this section refer to the present even though part of what's described here is not implemented yet.
|
||||
|
|
@ -51,7 +32,7 @@ otherwise outright broken. Feel free to report bugs!
|
|||
because incremental compilation is designed for modules and it doesn't play well with the interactive nature of a REPL session. To show the current state
|
||||
of the REPL, type `#show` (this will print all the code that has been typed so far), while to reset everything, type `#reset`. You can also type
|
||||
`#clear` if you want a clean slate to type in, but note that it won't reset the REPL state. If adding a new piece of code causes compilation to fail, the REPL will not add the last piece of code to the input so you can type it again and recompile without having to exit the program and start from scratch. You can move through the code using left/right arrows and go to a new line by pressing Ctrl+Enter. Using the up/down keys on your keyboard
|
||||
will move through the input history (which is never reset).
|
||||
will move through the input history (which is never reset). Also note that UTF-8 is currently unsupported in the REPL (it will be soon though!)
|
||||
|
||||
|
||||
**Disclaimer 3**: Currently, the `std` module has to be _always_ imported explicitly for even the most basic snippets to work. This is because intrinsic types and builtin operators are defined within it: if it is not imported, peon won't even know how to parse `2 + 2` (and even if it could, it would have no idea what the type of the expression would be). You can have a look at the [peon standard library](src/peon/stdlib) to see how the builtins are defined (be aware that they heavily rely on compiler black magic to work) and can even provide your own implementation if you're so inclined.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,29 @@
|
|||
# Peon design scratchpad
|
||||
|
||||
This is just a random doc I made to keep track of all the design changes I have
|
||||
in mind for Peon: with this being my first serious attempt at making a programming
|
||||
language that's actually _useful_, I want to get the design right the first time
|
||||
(no one wants to make JavaScript 2.0, right? _Right?_).
|
||||
|
||||
|
||||
The basic idea is:
|
||||
- Some peon code comes in (from a file or as command-line input, doesn't matter)
|
||||
- It gets tokenized and parsed into a typeless AST
|
||||
- The compiler processes the typeless AST into a typed one
|
||||
- The typed AST is passed to an optional optimizer module, which spits
|
||||
out another (potentially identical) typed AST representing the optimized
|
||||
program. The optimizer is always run even when optimizations are disabled,
|
||||
as it takes care of performing closure conversion and other cool stuff
|
||||
- The typed AST is passed to a code generator module that is specific to every
|
||||
backend/platform, which actually takes care of producing the code that will
|
||||
then be executed
|
||||
|
||||
|
||||
The current design is fairly modular and some parts of the codebase are more final
|
||||
than others: for example, the lexer and parser are more or less complete and unlikely
|
||||
to undergo massive changes in the future as opposed to the compiler which has been subject
|
||||
to many major refactoring steps as the project went along, but I digress.
|
||||
|
||||
The typed AST format should ideally be serializable to binary files so that I can slot in
|
||||
different optimizer/code generator modules written in different languages without the need
|
||||
to use FFI. The format will serve a similar purpose to the IR used by gcc (GIMPLE), but
|
||||
|
|
@ -145,28 +145,28 @@ proc reallocate*(self: var PeonVM, p: pointer, oldSize: int, newSize: int): poin
|
|||
quit(139) # For now, there's not much we can do if we can't get the memory we need, so we exit
|
||||
|
||||
|
||||
template resizeArray*(self: var PeonVM, kind: untyped, p: pointer, oldCount, newCount: int): untyped =
|
||||
template resizeArray(self: var PeonVM, kind: untyped, p: pointer, oldCount, newCount: int): untyped =
|
||||
## Handy template to resize a dynamic array
|
||||
cast[ptr UncheckedArray[kind]](reallocate(self, p, sizeof(kind) * oldCount, sizeof(kind) * newCount))
|
||||
|
||||
|
||||
template freeArray*(self: var PeonVM, kind: untyped, p: pointer, size: int): untyped =
|
||||
template freeArray(self: var PeonVM, kind: untyped, p: pointer, size: int): untyped =
|
||||
## Frees a dynamic array
|
||||
discard reallocate(self, p, sizeof(kind) * size, 0)
|
||||
|
||||
|
||||
template free*(self: var PeonVM, kind: typedesc, p: pointer): untyped =
|
||||
template free(self: var PeonVM, kind: typedesc, p: pointer): untyped =
|
||||
## Frees a pointer by reallocating its
|
||||
## size to 0
|
||||
discard reallocate(self, p, sizeof(kind), 0)
|
||||
|
||||
|
||||
template setKind*[T, K](t: var T, kind: untyped, target: K) =
|
||||
template setKind[T, K](t: var T, kind: untyped, target: K) =
|
||||
## Thanks to https://forum.nim-lang.org/t/8312
|
||||
cast[ptr K](cast[int](addr t) + offsetOf(typeof(t), kind))[] = target
|
||||
|
||||
|
||||
proc allocate*(self: var PeonVM, kind: ObjectKind, size: typedesc, count: int): ptr HeapObject {.inline.} =
|
||||
proc allocate(self: var PeonVM, kind: ObjectKind, size: typedesc, count: int): ptr HeapObject {.inline.} =
|
||||
## Allocates an object on the heap
|
||||
result = cast[ptr HeapObject](self.reallocate(nil, 0, sizeof(HeapObject)))
|
||||
setkind(result[], kind, kind)
|
||||
|
|
|
|||
|
|
@ -53,10 +53,12 @@ type
|
|||
PeonBackend* = enum
|
||||
## An enumeration of the peon backends
|
||||
Bytecode, NativeC, NativeCpp
|
||||
|
||||
PragmaKind* = enum
|
||||
## An enumeration of pragma types
|
||||
Immediate,
|
||||
Delayed
|
||||
|
||||
TypeKind* = enum
|
||||
## An enumeration of compile-time
|
||||
## types
|
||||
|
|
@ -65,6 +67,7 @@ type
|
|||
Char, Byte, String, Function, CustomType,
|
||||
Nil, Nan, Bool, Inf, Typevar, Generic,
|
||||
Reference, Pointer, Any, All, Union, Auto
|
||||
|
||||
Type* = ref object
|
||||
## A wrapper around
|
||||
## compile-time types
|
||||
|
|
@ -144,12 +147,28 @@ type
|
|||
# Who is this name exported to? (Only makes sense if isPrivate
|
||||
# equals false)
|
||||
exportedTo*: HashSet[Name]
|
||||
# Has the compiler generates this name internally or
|
||||
# Has the compiler generated this name internally or
|
||||
# does it come from user code?
|
||||
isReal*: bool
|
||||
# Is this name a builtin?
|
||||
isBuiltin*: bool
|
||||
|
||||
## BACKEND-SPECIFIC FIELDS
|
||||
|
||||
# Bytecode backend stuff
|
||||
|
||||
# For functions, this marks where their body's
|
||||
# code begins. For variables, it represents the
|
||||
# instruction after which they will be available
|
||||
# for use. Other name types don't use this value
|
||||
codePos*: int
|
||||
# The location of this name on the stack.
|
||||
# Only makes sense for names that actually
|
||||
# materialize on the call stack at runtime
|
||||
# (except for functions, where we use it to
|
||||
# signal where the function's frame starts)
|
||||
position*: int
|
||||
|
||||
WarningKind* {.pure.} = enum
|
||||
## A warning enumeration type
|
||||
UnreachableCode, UnusedName, ShadowOuterScope,
|
||||
|
|
@ -1025,7 +1044,7 @@ proc declare*(self: Compiler, node: ASTNode): Name {.discardable.} =
|
|||
continue
|
||||
if name.kind in [NameKind.Var, NameKind.Module, NameKind.CustomType, NameKind.Enum]:
|
||||
if name.depth < n.depth:
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' shadows a name from an outer scope ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
elif name.owner != n.owner:
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' shadows a name from an outer module ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' at depth {name.depth} shadows a name from an outer scope ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
if name.owner != n.owner:
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' at depth {name.depth} shadows a name from an outer module ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
return n
|
||||
|
|
|
|||
|
|
@ -40,21 +40,7 @@ type
|
|||
## An internal compiler function called
|
||||
## by pragmas
|
||||
kind: PragmaKind
|
||||
handler: proc (self: BytecodeCompiler, pragma: Pragma, name: BytecodeName)
|
||||
|
||||
BytecodeName = ref object of Name
|
||||
## A name object in our bytecode target
|
||||
# For functions, this marks where their body's
|
||||
# code begins. For variables, it represents the
|
||||
# instruction after which they will be available
|
||||
# for use. Other name types don't use this value
|
||||
codePos: int
|
||||
# The location of this name on the stack.
|
||||
# Only makes sense for names that actually
|
||||
# materialize on the call stack at runtime
|
||||
# (except for functions, where we use it to
|
||||
# signal where the function's frame starts)
|
||||
position: int
|
||||
handler: proc (self: BytecodeCompiler, pragma: Pragma, name: Name)
|
||||
|
||||
Loop = object
|
||||
## A "loop object" used
|
||||
|
|
@ -95,8 +81,8 @@ type
|
|||
# Stores the position of all jumps
|
||||
jumps: seq[tuple[patched: bool, offset: int]]
|
||||
# Metadata about function locations
|
||||
functions: seq[tuple[start, stop, pos: int, fn: BytecodeName]]
|
||||
forwarded: seq[tuple[name: BytecodeName, pos: int]]
|
||||
functions: seq[tuple[start, stop, pos: int, fn: Name]]
|
||||
forwarded: seq[tuple[name: Name, pos: int]]
|
||||
# The topmost occupied stack slot
|
||||
# in the current frame (0-indexed)
|
||||
stackIndex: int
|
||||
|
|
@ -108,18 +94,18 @@ proc compile*(self: BytecodeCompiler, ast: seq[Declaration], file: string, lines
|
|||
mode: CompileMode = Debug): Chunk
|
||||
proc statement(self: BytecodeCompiler, node: Statement)
|
||||
proc declaration(self: BytecodeCompiler, node: Declaration)
|
||||
proc varDecl(self: BytecodeCompiler, node: VarDecl, name: BytecodeName)
|
||||
proc varDecl(self: BytecodeCompiler, node: VarDecl, name: Name)
|
||||
proc specialize(self: BytecodeCompiler, typ: Type, args: seq[Expression]): Type {.discardable.}
|
||||
proc patchReturnAddress(self: BytecodeCompiler, pos: int)
|
||||
proc handleMagicPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName)
|
||||
proc handlePurePragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName)
|
||||
proc handleErrorPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName)
|
||||
method dispatchPragmas(self: BytecodeCompiler, name: BytecodeName)
|
||||
method dispatchDelayedPragmas(self: BytecodeCompiler, name: BytecodeName)
|
||||
proc funDecl(self: BytecodeCompiler, node: FunDecl, name: BytecodeName)
|
||||
proc compileModule(self: BytecodeCompiler, module: BytecodeName)
|
||||
proc generateCall(self: BytecodeCompiler, fn: BytecodeName, args: seq[Expression], line: int)
|
||||
method prepareFunction(self: BytecodeCompiler, fn: BytecodeName)
|
||||
proc handleMagicPragma(self: BytecodeCompiler, pragma: Pragma, name: Name)
|
||||
proc handlePurePragma(self: BytecodeCompiler, pragma: Pragma, name: Name)
|
||||
proc handleErrorPragma(self: BytecodeCompiler, pragma: Pragma, name: Name)
|
||||
method dispatchPragmas(self: BytecodeCompiler, name: Name)
|
||||
method dispatchDelayedPragmas(self: BytecodeCompiler, name: Name)
|
||||
proc funDecl(self: BytecodeCompiler, node: FunDecl, name: Name)
|
||||
proc compileModule(self: BytecodeCompiler, module: Name)
|
||||
proc generateCall(self: BytecodeCompiler, fn: Name, args: seq[Expression], line: int)
|
||||
method prepareFunction(self: BytecodeCompiler, fn: Name)
|
||||
# End of forward declarations
|
||||
|
||||
|
||||
|
|
@ -317,6 +303,7 @@ proc emitJump(self: BytecodeCompiler, opcode: OpCode, line: int): int =
|
|||
result = self.jumps.high()
|
||||
|
||||
|
||||
|
||||
proc fixFunctionOffsets(self: BytecodeCompiler, where, oldLen: int) =
|
||||
## Fixes function offsets after the size of our
|
||||
## bytecode has changed
|
||||
|
|
@ -387,8 +374,8 @@ proc fixNames(self: BytecodeCompiler, where, oldLen: int) =
|
|||
## after the size of the bytecode has changed
|
||||
let offset = self.chunk.code.len() - oldLen
|
||||
for name in self.names:
|
||||
if BytecodeName(name).codePos > where:
|
||||
BytecodeName(name).codePos += offset
|
||||
if name.codePos > where:
|
||||
name.codePos += offset
|
||||
if name.valueType.kind == Function:
|
||||
name.valueType.location += offset
|
||||
|
||||
|
|
@ -548,10 +535,10 @@ proc patchForwardDeclarations(self: BytecodeCompiler) =
|
|||
## Patches forward declarations and looks
|
||||
## for their implementations so that calls
|
||||
## to them work properly
|
||||
var impl: BytecodeName
|
||||
var impl: Name
|
||||
var pos: array[8, uint8]
|
||||
for (forwarded, position) in self.forwarded:
|
||||
impl = BytecodeName(self.match(forwarded.ident.token.lexeme, forwarded.valueType, allowFwd=false))
|
||||
impl = self.match(forwarded.ident.token.lexeme, forwarded.valueType, allowFwd=false)
|
||||
if position == 0:
|
||||
continue
|
||||
pos = impl.codePos.toLong()
|
||||
|
|
@ -693,153 +680,6 @@ proc unpackUnion(self: BytecodeCompiler, condition: Expression, list: var seq[tu
|
|||
self.error("invalid type constraint in type union", condition)
|
||||
|
||||
|
||||
proc declare(self: BytecodeCompiler, node: ASTNode): BytecodeName {.discardable.} =
|
||||
## Statically declares a name into the current scope.
|
||||
## "Declaring" a name only means updating our internal
|
||||
## list of identifiers so that further calls to resolve()
|
||||
## correctly return them. There is no code to actually
|
||||
## declare a variable at runtime: the value is already
|
||||
## on the stack
|
||||
var declaredName: string = ""
|
||||
var n: BytecodeName
|
||||
if self.names.high() > 16777215:
|
||||
# If someone ever hits this limit in real-world scenarios, I swear I'll
|
||||
# slap myself 100 times with a sign saying "I'm dumb". Mark my words
|
||||
self.error("cannot declare more than 16777215 names at a time")
|
||||
case node.kind:
|
||||
of NodeKind.varDecl:
|
||||
var node = VarDecl(node)
|
||||
declaredName = node.name.token.lexeme
|
||||
# Creates a new Name entry so that self.identifier emits the proper stack offset
|
||||
self.names.add(BytecodeName(depth: self.depth,
|
||||
ident: node.name,
|
||||
isPrivate: node.isPrivate,
|
||||
owner: self.currentModule,
|
||||
file: self.file,
|
||||
isConst: node.isConst,
|
||||
valueType: nil, # Done later
|
||||
isLet: node.isLet,
|
||||
line: node.token.line,
|
||||
belongsTo: self.currentFunction,
|
||||
kind: NameKind.Var,
|
||||
node: node,
|
||||
isReal: true
|
||||
))
|
||||
n = BytecodeName(self.names[^1])
|
||||
of NodeKind.funDecl:
|
||||
var node = FunDecl(node)
|
||||
declaredName = node.name.token.lexeme
|
||||
var fn = BytecodeName(depth: self.depth,
|
||||
isPrivate: node.isPrivate,
|
||||
isConst: false,
|
||||
owner: self.currentModule,
|
||||
file: self.file,
|
||||
valueType: Type(kind: Function,
|
||||
returnType: nil, # We check it later
|
||||
args: @[],
|
||||
fun: node,
|
||||
forwarded: node.body.isNil(),
|
||||
isAuto: false),
|
||||
ident: node.name,
|
||||
node: node,
|
||||
isLet: false,
|
||||
line: node.token.line,
|
||||
kind: NameKind.Function,
|
||||
belongsTo: self.currentFunction,
|
||||
isReal: true)
|
||||
if node.isTemplate:
|
||||
fn.valueType.compiled = true
|
||||
if node.generics.len() > 0:
|
||||
fn.isGeneric = true
|
||||
var typ: Type
|
||||
for argument in node.arguments:
|
||||
typ = self.infer(argument.valueType)
|
||||
if not typ.isNil() and typ.kind == Auto:
|
||||
fn.valueType.isAuto = true
|
||||
if fn.isGeneric:
|
||||
self.error("automatic types cannot be used within generics", argument.valueType)
|
||||
break
|
||||
typ = self.infer(node.returnType)
|
||||
if not typ.isNil() and typ.kind == Auto:
|
||||
fn.valueType.isAuto = true
|
||||
if fn.isGeneric:
|
||||
self.error("automatic types cannot be used within generics", node.returnType)
|
||||
self.names.add(fn)
|
||||
self.prepareFunction(BytecodeName(fn))
|
||||
n = BytecodeName(fn)
|
||||
of NodeKind.importStmt:
|
||||
var node = ImportStmt(node)
|
||||
# We change the name of the module internally so that
|
||||
# if you import /path/to/mod, then doing mod.f() will
|
||||
# still work without any extra work on our end. Note how
|
||||
# we don't change the metadata about the identifier's
|
||||
# position so that error messages still highlight the
|
||||
# full path
|
||||
let path = node.moduleName.token.lexeme
|
||||
node.moduleName.token.lexeme = node.moduleName.token.lexeme.extractFilename()
|
||||
self.names.add(BytecodeName(depth: self.depth,
|
||||
owner: self.currentModule,
|
||||
file: "", # The file of the module isn't known until it's compiled!
|
||||
path: path,
|
||||
ident: node.moduleName,
|
||||
line: node.moduleName.token.line,
|
||||
kind: NameKind.Module,
|
||||
isPrivate: false,
|
||||
isReal: true
|
||||
))
|
||||
n = BytecodeName(self.names[^1])
|
||||
declaredName = self.names[^1].ident.token.lexeme
|
||||
of NodeKind.typeDecl:
|
||||
var node = TypeDecl(node)
|
||||
self.names.add(BytecodeName(kind: NameKind.CustomType,
|
||||
depth: self.depth,
|
||||
owner: self.currentModule,
|
||||
node: node,
|
||||
ident: node.name,
|
||||
line: node.token.line,
|
||||
isPrivate: node.isPrivate,
|
||||
isReal: true,
|
||||
belongsTo: self.currentFunction
|
||||
)
|
||||
)
|
||||
n = BytecodeName(self.names[^1])
|
||||
declaredName = node.name.token.lexeme
|
||||
if node.value.isNil():
|
||||
discard # TODO: Fields
|
||||
else:
|
||||
case node.value.kind:
|
||||
of identExpr:
|
||||
n.valueType = self.inferOrError(node.value)
|
||||
of binaryExpr:
|
||||
# Type union
|
||||
n.valueType = Type(kind: Union, types: @[])
|
||||
self.unpackUnion(node.value, n.valueType.types)
|
||||
else:
|
||||
discard
|
||||
else:
|
||||
discard # TODO: enums
|
||||
if not n.isNil():
|
||||
self.dispatchPragmas(n)
|
||||
for name in self.findByName(declaredName):
|
||||
if name == n:
|
||||
continue
|
||||
# We don't check for name clashes with functions because self.match() does that
|
||||
if name.kind in [NameKind.Var, NameKind.Module, NameKind.CustomType, NameKind.Enum] and name.depth == n.depth and name.owner == n.owner:
|
||||
self.error(&"re-declaration of {declaredName} is not allowed (previously declared in {name.owner.ident.token.lexeme}:{name.ident.token.line}:{name.ident.token.relPos.start})")
|
||||
for name in self.names:
|
||||
if name == n:
|
||||
break
|
||||
if name.ident.token.lexeme != declaredName:
|
||||
continue
|
||||
if name.owner != n.owner and (name.isPrivate or n.owner notin name.exportedTo):
|
||||
continue
|
||||
if name.kind in [NameKind.Var, NameKind.Module, NameKind.CustomType, NameKind.Enum]:
|
||||
if name.depth < n.depth:
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' shadows a name from an outer scope ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
elif name.owner != n.owner:
|
||||
self.warning(WarningKind.ShadowOuterScope, &"'{declaredName}' shadows a name from an outer module ({name.owner.file}.pn:{name.ident.token.line}:{name.ident.token.relPos.start})", n)
|
||||
return n
|
||||
|
||||
|
||||
proc emitLoop(self: BytecodeCompiler, begin: int, line: int) =
|
||||
## Emits a JumpBackwards instruction with the correct
|
||||
|
|
@ -865,7 +705,7 @@ proc patchBreaks(self: BytecodeCompiler) =
|
|||
self.patchJump(brk)
|
||||
|
||||
|
||||
proc handleMagicPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName) =
|
||||
proc handleMagicPragma(self: BytecodeCompiler, pragma: Pragma, name: Name) =
|
||||
## Handles the "magic" pragma. Assumes the given name is already
|
||||
## declared
|
||||
if pragma.args.len() != 1:
|
||||
|
|
@ -886,7 +726,7 @@ proc handleMagicPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeNam
|
|||
self.error("'magic' pragma is not valid in this context")
|
||||
|
||||
|
||||
proc handleErrorPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName) =
|
||||
proc handleErrorPragma(self: BytecodeCompiler, pragma: Pragma, name: Name) =
|
||||
## Handles the "error" pragma
|
||||
if pragma.args.len() != 1:
|
||||
self.error("'error' pragma: wrong number of arguments")
|
||||
|
|
@ -897,7 +737,7 @@ proc handleErrorPragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeNam
|
|||
self.error(pragma.args[0].token.lexeme[1..^2])
|
||||
|
||||
|
||||
proc handlePurePragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName) =
|
||||
proc handlePurePragma(self: BytecodeCompiler, pragma: Pragma, name: Name) =
|
||||
## Handles the "pure" pragma
|
||||
case name.node.kind:
|
||||
of NodeKind.funDecl:
|
||||
|
|
@ -908,7 +748,7 @@ proc handlePurePragma(self: BytecodeCompiler, pragma: Pragma, name: BytecodeName
|
|||
self.error("'pure' pragma is not valid in this context")
|
||||
|
||||
|
||||
method dispatchPragmas(self: BytecodeCompiler, name: BytecodeName) =
|
||||
method dispatchPragmas(self: BytecodeCompiler, name: Name) =
|
||||
## Dispatches pragmas bound to objects
|
||||
if name.node.isNil():
|
||||
return
|
||||
|
|
@ -930,7 +770,7 @@ method dispatchPragmas(self: BytecodeCompiler, name: BytecodeName) =
|
|||
f.handler(self, pragma, name)
|
||||
|
||||
|
||||
method dispatchDelayedPragmas(self: BytecodeCompiler, name: BytecodeName) =
|
||||
method dispatchDelayedPragmas(self: BytecodeCompiler, name: Name) =
|
||||
## Dispatches pragmas bound to objects once they
|
||||
## are called. Only applies to functions
|
||||
if name.node.isNil():
|
||||
|
|
@ -983,7 +823,7 @@ proc generateCall(self: BytecodeCompiler, fn: Type, args: seq[Expression], line:
|
|||
self.patchReturnAddress(pos)
|
||||
|
||||
|
||||
method prepareFunction(self: BytecodeCompiler, fn: BytecodeName) =
|
||||
method prepareFunction(self: BytecodeCompiler, fn: Name) =
|
||||
## "Prepares" a function declaration by declaring
|
||||
## its arguments and typechecking it
|
||||
|
||||
|
|
@ -993,7 +833,7 @@ method prepareFunction(self: BytecodeCompiler, fn: BytecodeName) =
|
|||
var constraints: seq[tuple[match: bool, kind: Type]] = @[]
|
||||
for gen in fn.node.generics:
|
||||
self.unpackGenerics(gen.cond, constraints)
|
||||
self.names.add(BytecodeName(depth: fn.depth + 1,
|
||||
self.names.add(Name(depth: fn.depth + 1,
|
||||
isPrivate: true,
|
||||
valueType: Type(kind: Generic, name: gen.name.token.lexeme, cond: constraints),
|
||||
codePos: 0,
|
||||
|
|
@ -1015,7 +855,7 @@ method prepareFunction(self: BytecodeCompiler, fn: BytecodeName) =
|
|||
if self.names.high() > 16777215:
|
||||
self.error("cannot declare more than 16777215 variables at a time")
|
||||
inc(self.stackIndex)
|
||||
self.names.add(BytecodeName(depth: fn.depth + 1,
|
||||
self.names.add(Name(depth: fn.depth + 1,
|
||||
isPrivate: true,
|
||||
owner: fn.owner,
|
||||
file: fn.file,
|
||||
|
|
@ -1047,7 +887,7 @@ method prepareFunction(self: BytecodeCompiler, fn: BytecodeName) =
|
|||
fn.valueType.compiled = true
|
||||
|
||||
|
||||
proc prepareAutoFunction(self: BytecodeCompiler, fn: BytecodeName, args: seq[tuple[name: string, kind: Type, default: Expression]]): BytecodeName =
|
||||
proc prepareAutoFunction(self: BytecodeCompiler, fn: Name, args: seq[tuple[name: string, kind: Type, default: Expression]]): Name =
|
||||
## "Prepares" an automatic function declaration
|
||||
## by declaring a concrete version of it along
|
||||
## with its arguments
|
||||
|
|
@ -1069,7 +909,7 @@ proc prepareAutoFunction(self: BytecodeCompiler, fn: BytecodeName, args: seq[tup
|
|||
if self.names.high() > 16777215:
|
||||
self.error("cannot declare more than 16777215 variables at a time")
|
||||
inc(self.stackIndex)
|
||||
self.names.add(BytecodeName(depth: fn.depth + 1,
|
||||
self.names.add(Name(depth: fn.depth + 1,
|
||||
isPrivate: true,
|
||||
owner: fn.owner,
|
||||
file: fn.file,
|
||||
|
|
@ -1093,7 +933,7 @@ proc prepareAutoFunction(self: BytecodeCompiler, fn: BytecodeName, args: seq[tup
|
|||
return fn
|
||||
|
||||
|
||||
proc generateCall(self: BytecodeCompiler, fn: BytecodeName, args: seq[Expression], line: int) =
|
||||
proc generateCall(self: BytecodeCompiler, fn: Name, args: seq[Expression], line: int) =
|
||||
## Small wrapper that abstracts emitting a call instruction
|
||||
## for a given function
|
||||
self.dispatchDelayedPragmas(fn)
|
||||
|
|
@ -1167,18 +1007,18 @@ proc beginProgram(self: BytecodeCompiler): int =
|
|||
## entry point to be patched by terminateProgram
|
||||
if self.currentModule.isNil():
|
||||
# We declare the program's main module
|
||||
var mainModule = BytecodeName(kind: NameKind.Module,
|
||||
depth: 0,
|
||||
isPrivate: true,
|
||||
isConst: false,
|
||||
isLet: false,
|
||||
owner: nil,
|
||||
file: self.file,
|
||||
path: self.file,
|
||||
codePos: 0,
|
||||
ident: newIdentExpr(Token(lexeme: self.file, kind: Identifier)),
|
||||
resolved: true,
|
||||
line: 1)
|
||||
var mainModule = Name(kind: NameKind.Module,
|
||||
depth: 0,
|
||||
isPrivate: true,
|
||||
isConst: false,
|
||||
isLet: false,
|
||||
owner: nil,
|
||||
file: self.file,
|
||||
path: self.file,
|
||||
codePos: 0,
|
||||
ident: newIdentExpr(Token(lexeme: self.file, kind: Identifier)),
|
||||
resolved: true,
|
||||
line: 1)
|
||||
self.names.add(mainModule)
|
||||
self.currentModule = mainModule
|
||||
# Every peon program has a hidden entry point in
|
||||
|
|
@ -1191,7 +1031,7 @@ proc beginProgram(self: BytecodeCompiler): int =
|
|||
# of where our program ends (which we don't know yet).
|
||||
# To fix this, we emit dummy offsets and patch them
|
||||
# later, once we know the boundaries of our hidden main()
|
||||
var main = BytecodeName(depth: 0,
|
||||
var main = Name(depth: 0,
|
||||
isPrivate: true,
|
||||
isConst: false,
|
||||
isLet: false,
|
||||
|
|
@ -1326,7 +1166,7 @@ method unary(self: BytecodeCompiler, node: UnaryExpr, compile: bool = true): Typ
|
|||
result = self.specialize(result, @[node.a])
|
||||
result = result.returnType
|
||||
if compile:
|
||||
self.generateCall(BytecodeName(impl), @[node.a], impl.line)
|
||||
self.generateCall(impl, @[node.a], impl.line)
|
||||
|
||||
|
||||
method binary(self: BytecodeCompiler, node: BinaryExpr, compile: bool = true): Type {.discardable.} =
|
||||
|
|
@ -1339,7 +1179,7 @@ method binary(self: BytecodeCompiler, node: BinaryExpr, compile: bool = true): T
|
|||
result = self.specialize(result, @[node.a, node.b])
|
||||
result = result.returnType
|
||||
if compile:
|
||||
self.generateCall(BytecodeName(impl), @[node.a, node.b], impl.line)
|
||||
self.generateCall(impl, @[node.a, node.b], impl.line)
|
||||
|
||||
|
||||
method identifier(self: BytecodeCompiler, node: IdentExpr, name: Name = nil, compile: bool = true, strict: bool = true): Type {.discardable.} =
|
||||
|
|
@ -1347,9 +1187,9 @@ method identifier(self: BytecodeCompiler, node: IdentExpr, name: Name = nil, com
|
|||
var s = name
|
||||
if s.isNil():
|
||||
if strict:
|
||||
s = BytecodeName(self.resolveOrError(node))
|
||||
s = self.resolveOrError(node)
|
||||
else:
|
||||
s = BytecodeName(self.resolve(node))
|
||||
s = self.resolve(node)
|
||||
if s.isNil() and not strict:
|
||||
return nil
|
||||
result = s.valueType
|
||||
|
|
@ -1384,7 +1224,7 @@ method identifier(self: BytecodeCompiler, node: IdentExpr, name: Name = nil, com
|
|||
# Loads a regular variable from the current frame
|
||||
self.emitByte(LoadVar, s.ident.token.line)
|
||||
# No need to check for -1 here: we already did a nil check above!
|
||||
self.emitBytes(BytecodeName(s).position.toTriple(), s.ident.token.line)
|
||||
self.emitBytes(s.position.toTriple(), s.ident.token.line)
|
||||
|
||||
|
||||
method assignment(self: BytecodeCompiler, node: ASTNode, compile: bool = true): Type {.discardable.} =
|
||||
|
|
@ -1393,7 +1233,7 @@ method assignment(self: BytecodeCompiler, node: ASTNode, compile: bool = true):
|
|||
of assignExpr:
|
||||
let node = AssignExpr(node)
|
||||
let name = IdentExpr(node.name)
|
||||
var r = BytecodeName(self.resolveOrError(name))
|
||||
var r = self.resolveOrError(name)
|
||||
if r.isConst:
|
||||
self.error(&"cannot assign to '{name.token.lexeme}' (value is a constant)", name)
|
||||
elif r.isLet:
|
||||
|
|
@ -1452,10 +1292,10 @@ method call(self: BytecodeCompiler, node: CallExpr, compile: bool = true): Type
|
|||
if impl.isGeneric:
|
||||
result = self.specialize(result, argExpr)
|
||||
if impl.valueType.isAuto:
|
||||
impl = self.prepareAutoFunction(BytecodeName(impl), args)
|
||||
impl = self.prepareAutoFunction(impl, args)
|
||||
result = impl.valueType
|
||||
if not impl.valueType.compiled:
|
||||
self.funDecl(FunDecl(result.fun), BytecodeName(impl))
|
||||
self.funDecl(FunDecl(result.fun), impl)
|
||||
result = result.returnType
|
||||
if compile:
|
||||
if impl.valueType.fun.kind == funDecl and FunDecl(impl.valueType.fun).isTemplate:
|
||||
|
|
@ -1474,7 +1314,7 @@ method call(self: BytecodeCompiler, node: CallExpr, compile: bool = true): Type
|
|||
else:
|
||||
self.declaration(decl)
|
||||
else:
|
||||
self.generateCall(BytecodeName(impl), argExpr, node.token.line)
|
||||
self.generateCall(impl, argExpr, node.token.line)
|
||||
of NodeKind.callExpr:
|
||||
# Calling a call expression, like hello()()
|
||||
var node: Expression = node
|
||||
|
|
@ -1500,7 +1340,7 @@ method call(self: BytecodeCompiler, node: CallExpr, compile: bool = true): Type
|
|||
if impl.isGeneric:
|
||||
result = self.specialize(result, argExpr)
|
||||
result = result.returnType
|
||||
self.generateCall(BytecodeName(impl), argExpr, node.token.line)
|
||||
self.generateCall(impl, argExpr, node.token.line)
|
||||
# TODO: Calling lambdas on-the-fly (i.e. on the same line)
|
||||
else:
|
||||
let typ = self.infer(node)
|
||||
|
|
@ -1536,7 +1376,7 @@ method getItemExpr(self: BytecodeCompiler, node: GetItemExpr, compile: bool = tr
|
|||
else:
|
||||
self.error(&"ambiguous reference for '{node.name.token.lexeme}' in module '{name.ident.token.lexeme}'")
|
||||
if compile:
|
||||
self.identifier(nil, BytecodeName(values[0]))
|
||||
self.identifier(nil, values[0])
|
||||
else:
|
||||
self.error("invalid syntax", node.obj)
|
||||
else:
|
||||
|
|
@ -1550,7 +1390,7 @@ method lambdaExpr(self: BytecodeCompiler, node: LambdaExpr, compile: bool = true
|
|||
var constraints: seq[tuple[match: bool, kind: Type]] = @[]
|
||||
for gen in node.generics:
|
||||
self.unpackGenerics(gen.cond, constraints)
|
||||
self.names.add(BytecodeName(depth: self.depth,
|
||||
self.names.add(Name(depth: self.depth,
|
||||
isPrivate: true,
|
||||
valueType: Type(kind: Generic, name: gen.name.token.lexeme, cond: constraints),
|
||||
codePos: 0,
|
||||
|
|
@ -1562,12 +1402,12 @@ method lambdaExpr(self: BytecodeCompiler, node: LambdaExpr, compile: bool = true
|
|||
file: self.file))
|
||||
constraints = @[]
|
||||
var default: Expression
|
||||
var name: BytecodeName
|
||||
var name: Name
|
||||
var i = 0
|
||||
for argument in node.arguments:
|
||||
if self.names.high() > 16777215:
|
||||
self.error("cannot declare more than 16777215 variables at a time")
|
||||
name = BytecodeName(depth: self.depth,
|
||||
name = Name(depth: self.depth,
|
||||
isPrivate: true,
|
||||
owner: self.currentModule,
|
||||
file: self.currentModule.file,
|
||||
|
|
@ -1770,7 +1610,7 @@ proc importStmt(self: BytecodeCompiler, node: ImportStmt, compile: bool = true)
|
|||
var module = self.names[^1]
|
||||
try:
|
||||
if compile:
|
||||
self.compileModule(BytecodeName(module))
|
||||
self.compileModule(module)
|
||||
# Importing a module automatically exports
|
||||
# its public names to us
|
||||
for name in self.findInModule("", module):
|
||||
|
|
@ -1863,13 +1703,13 @@ proc switchStmt(self: BytecodeCompiler, node: SwitchStmt) =
|
|||
var ifJump: int = -1
|
||||
var thenJumps: seq[int] = @[]
|
||||
var fn: Type
|
||||
var impl: BytecodeName
|
||||
var impl: Name
|
||||
var default: Expression
|
||||
for branch in node.branches:
|
||||
self.emitByte(DupTop, branch.body.token.line)
|
||||
self.expression(branch.cond)
|
||||
fn = Type(kind: Function, returnType: Type(kind: Bool), args: @[("", typeOfA, default), ("", self.inferOrError(branch.cond), default)])
|
||||
impl = BytecodeName(self.match("==", fn, node))
|
||||
impl = self.match("==", fn, node)
|
||||
self.generateCall(impl, @[node.switch, branch.cond], impl.line)
|
||||
ifJump = self.emitJump(JumpIfFalsePop, branch.body.token.line)
|
||||
self.blockStmt(branch.body)
|
||||
|
|
@ -1942,7 +1782,7 @@ proc statement(self: BytecodeCompiler, node: Statement) =
|
|||
self.expression(Expression(node))
|
||||
|
||||
|
||||
proc varDecl(self: BytecodeCompiler, node: VarDecl, name: BytecodeName) =
|
||||
proc varDecl(self: BytecodeCompiler, node: VarDecl, name: Name) =
|
||||
## Compiles variable declarations
|
||||
var typ: Type
|
||||
# Our parser guarantees that the variable declaration
|
||||
|
|
@ -1975,7 +1815,7 @@ proc varDecl(self: BytecodeCompiler, node: VarDecl, name: BytecodeName) =
|
|||
name.valueType = typ
|
||||
|
||||
|
||||
proc funDecl(self: BytecodeCompiler, node: FunDecl, name: BytecodeName) =
|
||||
proc funDecl(self: BytecodeCompiler, node: FunDecl, name: Name) =
|
||||
## Compiles function declarations
|
||||
if node.token.kind == Operator and node.name.token.lexeme in [".", "="]:
|
||||
self.error(&"Due to compiler limitations, the '{node.name.token.lexeme}' operator cannot be currently overridden", node.name)
|
||||
|
|
@ -2142,7 +1982,7 @@ proc compile*(self: BytecodeCompiler, ast: seq[Declaration], file: string, lines
|
|||
result = self.chunk
|
||||
|
||||
|
||||
proc compileModule(self: BytecodeCompiler, module: BytecodeName) =
|
||||
proc compileModule(self: BytecodeCompiler, module: Name) =
|
||||
## Compiles an imported module into an existing chunk
|
||||
## using the compiler's internal parser and lexer objects
|
||||
var path = ""
|
||||
|
|
|
|||
12
src/main.nim
12
src/main.nim
|
|
@ -57,7 +57,7 @@ proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: Comp
|
|||
keep = true
|
||||
tokens: seq[Token] = @[]
|
||||
tree: seq[Declaration] = @[]
|
||||
compiler = newBytecodeCompiler()
|
||||
compiler = newBytecodeCompiler(replMode=true)
|
||||
compiled: Chunk
|
||||
serialized: Serialized
|
||||
tokenizer = newLexer()
|
||||
|
|
@ -70,7 +70,6 @@ proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: Comp
|
|||
tokenizer.fillSymbolTable()
|
||||
editor.bindEvent(jeQuit):
|
||||
stdout.styledWriteLine(fgGreen, "Goodbye!")
|
||||
editor.prompt = "=> "
|
||||
keep = false
|
||||
input = ""
|
||||
current = ""
|
||||
|
|
@ -83,6 +82,7 @@ proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: Comp
|
|||
input = editor.read()
|
||||
if input == "#reset":
|
||||
compiled = newChunk()
|
||||
compiler = newBytecodeCompiler(replMode=true)
|
||||
current = ""
|
||||
continue
|
||||
elif input == "#show":
|
||||
|
|
@ -137,8 +137,8 @@ proc repl(warnings: seq[WarningKind] = @[], mismatches: bool = false, mode: Comp
|
|||
styledEcho fgGreen, "OK"
|
||||
else:
|
||||
styledEcho fgRed, "Corrupted"
|
||||
stdout.styledWrite(fgBlue, "\t- CFI segment: ")
|
||||
if serialized.chunk.cfi == compiled.cfi:
|
||||
stdout.styledWrite(fgBlue, "\t- Functions segment: ")
|
||||
if serialized.chunk.functions == compiled.functions:
|
||||
styledEcho fgGreen, "OK"
|
||||
else:
|
||||
styledEcho fgRed, "Corrupted"
|
||||
|
|
@ -247,8 +247,8 @@ proc runFile(f: string, fromString: bool = false, dump: bool = true, breakpoints
|
|||
styledEcho fgGreen, "OK"
|
||||
else:
|
||||
styledEcho fgRed, "Corrupted"
|
||||
stdout.styledWrite(fgBlue, "\t- CFI segment: ")
|
||||
if serialized.chunk.cfi == compiled.cfi:
|
||||
stdout.styledWrite(fgBlue, "\t- Functions segment: ")
|
||||
if serialized.chunk.functions == compiled.functions:
|
||||
styledEcho fgGreen, "OK"
|
||||
else:
|
||||
styledEcho fgRed, "Corrupted"
|
||||
|
|
|
|||
|
|
@ -2,31 +2,31 @@
|
|||
import values;
|
||||
|
||||
|
||||
operator `&`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a, b: T): T {
|
||||
operator `&`*[T: Integer](a, b: T): T {
|
||||
#pragma[magic: "And", pure]
|
||||
}
|
||||
|
||||
|
||||
operator `|`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a, b: T): T {
|
||||
operator `|`*[T: Integer](a, b: T): T {
|
||||
#pragma[magic: "Or", pure]
|
||||
}
|
||||
|
||||
|
||||
operator `~`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a: T): T {
|
||||
operator `~`*[T: Integer](a: T): T {
|
||||
#pragma[magic: "Not", pure]
|
||||
}
|
||||
|
||||
|
||||
operator `>>`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a, b: T): T {
|
||||
operator `>>`*[T: Integer](a, b: T): T {
|
||||
#pragma[magic: "RShift", pure]
|
||||
}
|
||||
|
||||
|
||||
operator `<<`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a, b: T): T {
|
||||
operator `<<`*[T: Integer](a, b: T): T {
|
||||
#pragma[magic: "LShift", pure]
|
||||
}
|
||||
|
||||
|
||||
operator `^`*[T: int | uint64 | int32 | uint32 | int16 | uint16 | int8 | uint8](a, b: T): T {
|
||||
operator `^`*[T: Integer](a, b: T): T {
|
||||
#pragma[magic: "Xor", pure]
|
||||
}
|
||||
Loading…
Reference in New Issue