This commit is contained in:
nocturn9x 2021-01-11 08:07:02 +01:00
commit c79095c99b
24 changed files with 658 additions and 175 deletions

8
.gitignore vendored
View File

@ -13,10 +13,15 @@ htmldocs/
src/japl
src/compiler
tests/runtests
tests/maketest
src/lexer
src/vm
tests/runtests
testresults.txt
.testoutput.txt
.tempcode.jpl
.tempcode_drEHdZuwNYLqsQaMDMqeNRtmqoqXBXfnCfeqEcmcUYJToBVQkF.jpl
.tempoutput.txt
# MacOS
@ -36,6 +41,3 @@ main
config.nim
# test results
testresults.txt

136
README.md
View File

@ -1,150 +1,130 @@
# JAPL - Just Another Programming Language
JAPL is an interpreted, dynamically-typed, garbage-collected, and minimalistic programming language with C- and Java-like syntax.
# J.. what?
## J.. what?
You may wonder what's the meaning of JAPL: well, it turns out to be an acronym
for __Just Another Programming Language__, but beware! Despite the name, the pronunciation is the same as "JPL".
You may wonder what's the meaning of JAPL: well, it turns out to be an acronym for __Just Another Programming Language__, but beware! Despite the name, the pronunciation is the same as "JPL".
## Disclaimer
### Disclaimer
This project is currently a WIP (Work in Progress) and is not optimized nor complete.
The design of the language may change at any moment and all the source inside this repo
is alpha code quality, for now.
The design of the language may change at any moment and all the source inside this repo is alpha code quality, for now.
For other useful information, check the LICENSE file in this repo.
For other useful information, check the LICENSE file in this repo.
JAPL is licensed under the Apache 2.0 license.
## Project roadmap
### Project roadmap
In no particular order, here's a list that is constantly updated and that helps us to keep track
of what's done in JAPL:
- Parsing/Lexing :heavy_check_mark:
- Object oriented type system :heavy_check_mark:
- Control flow (if/else) :heavy_check_mark:
- Loops (for/while) :heavy_check_mark:
- Basic comparisons operators (`>`, `<`, `>=`, `<=`, `!=`, `==`) :heavy_check_mark:
- Logical operators (`!`, `or`, `and`) (:heavy_check_mark:)
- Multi-line comments `/* like this */` (can be nested) :heavy_check_mark:
- `inf` and `nan` types :heavy_check_mark:
- Basic arithmetic (`+`, `-`, `/`, `*`) :heavy_check_mark:
- Modulo division (`%`) and exponentiation (`**`) :heavy_check_mark:
- Bitwise operators (AND, OR, XOR, NOT) :heavy_check_mark:
- Global and local variables :heavy_check_mark:
- Explicit scopes using brackets :heavy_check_mark:
- Simple optimizations (constant string interning, singletons caching) :heavy_check_mark:
- Garbage collector (__Coming soon__)
- String slicing, with start:end syntax as well :heavy_check_mark:
- Operations on strings (addition, multiplication) :heavy_check_mark:
- Functions and Closures (__WIP__)
- Functions default and keyword arguments (__WIP__)
- An OOP system (class-based) (__Coming soon__)
- Builtins as classes (types) (__Coming soon__)
- A proper import system (__Coming soon__)
- Native asynchronous (`await`/`async fun`) support (__Coming soon__)
- Bytecode optimizations such as constant folding and stack caching (__Coming Soon__)
- Arbitrary-precision arithmetic (__Coming soon__)
- Generators (__Coming soon__)
- A standard library with collections, I/O utilities, scientific modules, etc (__Coming soon__)
- Multithreading and multiprocessing support with a global VM Lock like CPython (__Coming soon__)
- Exceptions (__Coming soon__)
- Optional JIT Compilation (__Coming soon__)
- [x] Parsing/Lexing
- [x] Object oriented type system
- [x] Control flow (if/else)
- [x] Loops (for/while)
- [x] Basic comparisons operators (`>`, `<`, `>=`, `<=`, `!=`, `==`)
- [x] Logical operators (`!`, `or`, `and`)
- [x] Multi-line comments `/* like this */` (can be nested)
- [x] `inf` and `nan` types
- [x] Basic arithmetic (`+`, `-`, `/`, `*`)
- [x] Modulo division (`%`) and exponentiation (`**`)
- [x] Bitwise operators (AND, OR, XOR, NOT)
- [x] Global and local variables
- [x] Explicit scopes using bracket
- [x] Simple optimizations (constant string interning, singletons caching
- [ ] Garbage collector
- [x] String slicing, with start:end syntax as well
- [x] Operations on strings (addition, multiplication)
- [ ] Functions and Closures
- [ ] Functions default and keyword arguments
- [ ] An OOP system (class-based)
- [ ] Builtins as classes (types)
- [ ] A proper import system
- [ ] Native asynchronous (`await`/`async fun`) support
- [ ] Bytecode optimizations such as constant folding and stack caching
- [ ] Arbitrary-precision arithmetic
- [ ] Generators
- [ ] A standard library with collections, I/O utilities, scientific modules, etc
- [ ] Multithreading and multiprocessing support with a global VM Lock like CPython
- [ ] Exceptions
- [ ] Optional JIT Compilation
### Classifiers
- __WIP__: Work In Progress, being implemented right now
- __Coming Soon__: Not yet implemented/designed but scheduled
- __Rework Needed__: The feature works, but can (and must) be optimized/reimplemented properly
- :heavy_check_mark:: The feature works as intended
- [x] : The feature works as intended
## Contributing
If you want to contribute, feel free to open a PR!
Right now there are some major issues with the virtual machine which need to be addressed
before the development can proceed, and some help is ~~desperately needed~~ greatly appreciated!
Right now there are some major issues with the virtual machine which need to be addressed before the development can proceed, and some help is ~~desperately needed~~ greatly appreciated!
To get started, you might want to have a look at the currently open issues and start from there
### Community
## Community
Our first goal is to create a welcoming and helpful community, so if you are so inclined, you might want to join our [Discord server](https://discord.gg/P8FYZvM) and our [forum](https://forum.japl-lang.com)! We can't wait to welcome you into our community :D
Our first goal is to create a welcoming and helpful community, so if you are so inclined,
you might want to join our [Discord server](https://discord.gg/P8FYZvM) and our [forum](https://forum.japl-lang.com)! We can't wait to welcome you into
our community :D
### A special thanks
JAPL is born thanks to the amazing work of Bob Nystrom that wrote a book available completely for free at [this](https://craftinginterpreters.com) link, where he describes the implementation of a simple language called Lox.
## A special thanks
JAPL is born thanks to the amazing work of Bob Nystrom that wrote a book available completely for free
at [this](https://craftinginterpreters.com) link, where he describes the implementation of a simple language called Lox.
# JAPL - Installing
## JAPL - Installing
JAPL is currently in its early stages and is therefore in a state of high mutability, so this installation guide might
not be always up to date.
## Requirements
### Requirements
To compile JAPL, you need the following:
- Nim >= 1.2 installed on your system
- Git (to clone the repository)
- Python >= 3.6 (Build script)
## Cloning the repo
### Cloning the repo
Once you've installed all the required tooling, you can clone the repo with the following command
```
```bash
git clone https://github.com/japl-lang/japl
```
## Running the build script
### Running the build script
As a next step, you need to run the build script. This will generate the required configuration files,
compile the JAPL runtime and run tests. There are some options that
can be tweaked with command-line options, for more information, run `python3 build.py --help`.
As a next step, you need to run the build script. This will generate the required configuration files, compile the JAPL runtime and run tests (unless `--skip-tests` is passed). There are some options that can be tweaked with command-line options, for more information, run `python3 build.py --help`.
To compile the JAPL runtime, you'll first need to move into the project's directory you cloned before, so run `cd japl`, then `python3 build.py ./src` and wait for it to complete. You should now find an executable named `japl` (or `japl.exe` on windows) inside the `src` folder.
To compile the JAPL runtime, you'll first need to move into the project's directory you cloned before,
so run `cd japl`, then `python3 build.py ./src` and wait for it to complete. You should now find an
executable named `japl` (or `japl.exe` on windows) inside the `src` folder.
If you're running under windows, you might encounter some issues when using forward-slashes as opposed to back-slashes in paths,
so you should replace `./src` with `.\src`
If you're running under windows, you might encounter some issues when using forward-slashes as opposed to back-slashes in paths, so you should replace `./src` with `.\src`
If you're running under linux, you can also call the build script with `./build.py` (assuming python is installed in the directory indicated by the shebang at the top of the file)
## Advanced builds
### Advanced builds
If you need more customizability or want to enable debugging for JAPL, there's a few things you can do.
### Nim compiler options
The build tool calls the system's nim compiler to build JAPL and by default, the only extra flag that's passed
to it is `--gc:markAndSweep`. If you want to customize the options passed to the compiler, you can pass a comma
separated list of key:value options (spaces are not allowed). For example, doing `python3 build.py src --flags d:release,threads:on`
will call `nim compile src/japl --gc:markAndSweep -d:release --threads:on`.
The build tool calls the system's nim compiler to build JAPL and by default, the only extra flag that's passed to it is `--gc:markAndSweep`. If you want to customize the options passed to the compiler, you can pass a comma separated list of key:value options (spaces are not allowed). For example, doing `python3 build.py src --flags d:release,threads:on` will call `nim compile src/japl --gc:markAndSweep -d:release --threads:on`.
### JAPL Debugging options
JAPL has some (still very beta) internal tooling to debug various parts of its ecosystem (compiler, runtime, GC, etc).
There are also some compile-time constants (such as the heap grow factor for the garbage collector) that can be set via the
`--options` parameter in the same fashion as the nim's compiler options. The available options are:
There are also some compile-time constants (such as the heap grow factor for the garbage collector) that can be set via the `--options` parameter in the same fashion as the nim's compiler options. The available options are:
- `debug_vm` -> Debugs the runtime, instruction by instruction, showing the effects of the bytecode on the VM's stack and scopes in real time (beware of bugs!)
- `debug_gc` -> Debugs the garbage collector (once we have one)
- `debug_alloc` -> Debugs memory allocation/deallocation
- `debug_compiler` -> Debugs the compiler, showing each bytes that is spit into the bytecode
- `debug_compiler` -> Debugs the compiler, showing each byte that is spit into the bytecode
Each of these options is independent of the others and can be enabled/disabled at will. To enable an option, pass `option_name:true` to `--options` while to disable it, replace `true` with `false`.
Note that the build tool will generate a file named `config.nim` inside the `src` directory and will use that for subsequent builds, so if you want to override it you'll have to pass `--override-config` as a command-line options. Passing it without any option will fallback to (somewhat) sensible defaults
**P.S.**: The test suite assumes that all debugging options are turned off, so for development/debug builds we recommend skipping the test suite by passing `--skip-tests` to the build script

View File

@ -46,7 +46,7 @@ import strformat
const MAP_LOAD_FACTOR* = {map_load_factor} # Load factor for builtin hashmaps (TODO)
const ARRAY_GROW_FACTOR* = {array_grow_factor} # How much extra memory to allocate for dynamic arrays (TODO)
const FRAMES_MAX* = {frames_max} # TODO: Inspect why the VM crashes if this exceeds this value
const FRAMES_MAX* = {frames_max} # The maximum recursion limit
const JAPL_VERSION* = "0.3.0"
const JAPL_RELEASE* = "alpha"
const DEBUG_TRACE_VM* = {debug_vm} # Traces VM execution
@ -72,6 +72,7 @@ Command-line options
-h, --help -> Show this help text and exit
-v, --version -> Print the JAPL version number and exit
-c -> Executes the passed string
"""'''
@ -97,7 +98,7 @@ def build(path: str, flags: Dict[str, str] = {}, options: Dict[str, bool] = {},
config_path = os.path.join(path, "config.nim")
main_path = os.path.join(path, "japl.nim")
logging.info("Just Another Build Tool, version 0.2")
logging.info("Just Another Build Tool, version 0.3.1")
if not os.path.exists(path):
logging.error(f"Input path '{path}' does not exist")
return
@ -150,16 +151,13 @@ def build(path: str, flags: Dict[str, str] = {}, options: Dict[str, bool] = {},
start = time()
try:
# TODO: Find a better way of running the test suite
process = run(f"{tests_path}", stdout=PIPE, stderr=PIPE, shell=True)
process = run(f"{tests_path}", shell=True, stderr=PIPE)
stderr = process.stderr.decode()
assert process.returncode == 0, f"Command '{command}' exited with non-0 exit code {process.returncode}, output below:\n{stderr}"
except Exception as fatal:
logging.error(f"A fatal unhandled exception occurred -> {type(fatal).__name__}: {fatal}")
else:
logging.debug(f"Test suite ran in {time() - start:.2f} seconds")
# This way it *looks* like we're running it now when it
# actually already happened
print(process.stdout.decode().rstrip("\n"))
logging.info("Test suite completed!")
@ -171,7 +169,7 @@ if __name__ == "__main__":
parser.add_argument("--options", help="Set compile-time options and constants, pass a comma-separated list of name:value (without spaces). "
"Note that if a config.nim file exists in the destination directory, that will override any setting defined here unless --override-config is used")
parser.add_argument("--override-config", help="Overrides the setting of an already existing config.nim file in the destination directory", action="store_true")
parser.add_argument("--skip-tests", help="Skips running the JAPL test suite", action="store_true")
parser.add_argument("--skip-tests", help="Skips running the JAPL test suite, useful for debug builds", action="store_true")
args = parser.parse_args()
flags = {
"gc": "markAndSweep",

33
resources/japl.nanorc Normal file
View File

@ -0,0 +1,33 @@
## Syntax highlighting for JAPL.
syntax python "\.jpl$"
header "^#!.*japl"
magic "JAPL script"
comment "//"
# Function definitions.
color brightblue "fun [0-9A-Za-z_]+"
# Methods definitions
color brightblue "method [0-9A-Za-z_]+"
# Keywords.
color brightcyan "\<(and|as|var|assert|async|await|break|class|continue)\>"
color brightcyan "\<(fun|del|elif|else|except|finally|for|from)\>"
color brightcyan "\<(global|if|import|in|is|lambda|nonlocal|not|or)\>"
color brightcyan "\<(pass|raise|return|try|while|with|yield)\>"
# Special values.
color brightmagenta "\<(false|true|nil|inf|nan)\>"
# Mono-quoted strings.
color brightgreen "'([^'\]|\\.)*'|'''"
color brightgreen ""([^"\]|\\.)*"|""""
color normal "'''|""""
# Comments.
color brightred "//.*"
color brightblue start="/\*" end="\*/"
# Reminders.
color brightwhite,yellow "\<(FIXME|TODO)\>"

View File

@ -52,50 +52,51 @@ proc repl() =
echo &"[Nim {NimVersion} on {hostOs} ({hostCPU})]"
continue
elif source != "":
var result = bytecodeVM.interpret(source, true, "stdin")
when DEBUG_TRACE_VM:
let result = bytecodeVM.interpret(source, true, "stdin")
echo &"Result: {result}"
when not DEBUG_TRACE_VM:
discard bytecodeVM.interpret(source, true, "stdin")
when DEBUG_TRACE_VM:
echo "==== Debugger exits ===="
proc main(file: string = "") =
proc main(file: var string = "", fromString: bool = false) =
var source: string
if file == "":
repl()
else:
elif not fromString:
var sourceFile: File
try:
sourceFile = open(filename=file)
except IOError:
echo &"Error: '{file}' could not be opened, probably the file doesn't exist or you don't have permission to read it"
return
var source: string
try:
source = readAll(sourceFile)
except IOError:
echo &"Error: '{file}' could not be read, probably you don't have the permission to read it"
var bytecodeVM = initVM()
bytecodeVM.stdlibInit()
when DEBUG_TRACE_VM:
echo "Debugger enabled, expect verbose output\n"
echo "==== VM Constants ====\n"
echo &"- FRAMES_MAX -> {FRAMES_MAX}"
echo "==== Code starts ====\n"
var result = bytecodeVM.interpret(source, false, file)
bytecodeVM.freeVM()
when DEBUG_TRACE_VM:
echo &"Result: {result}"
when DEBUG_TRACE_VM:
echo "==== Code ends ===="
else:
source = file
file = "<string>"
var bytecodeVM = initVM()
bytecodeVM.stdlibInit()
when DEBUG_TRACE_VM:
echo "Debugger enabled, expect verbose output\n"
echo "==== VM Constants ====\n"
echo &"- FRAMES_MAX -> {FRAMES_MAX}"
echo "==== Code starts ====\n"
let result = bytecodeVM.interpret(source, false, file)
echo &"Result: {result}"
when not DEBUG_TRACE_VM:
discard bytecodeVM.interpret(source, false, file)
bytecodeVM.freeVM()
when isMainModule:
var optParser = initOptParser(commandLineParams())
var file: string = ""
if paramCount() > 0:
if paramCount() notin 1..<2:
echo "usage: japl [filename]"
quit()
var fromString: bool = false
for kind, key, value in optParser.getopt():
case kind:
of cmdArgument:
@ -119,11 +120,14 @@ when isMainModule:
of "v":
echo JAPL_VERSION_STRING
quit()
of "c":
file = key
fromString = true
else:
echo &"error: unkown option '{key}'"
quit()
quit()
else:
echo "usage: japl [filename]"
quit()
main(file)
main(file, fromString)

View File

@ -204,6 +204,9 @@ proc scanToken(self: var Lexer) =
return
elif single == '\n':
self.line += 1
# TODO: Fix this to only emit semicolons where needed
# if self.tokens[^1].kind != TokenType.SEMICOLON:
# self.tokens.add(self.createToken(TOKENS[';']))
elif single in ['"', '\'']:
self.parseString(single)
elif single.isDigit():
@ -243,6 +246,8 @@ proc lex*(self: var Lexer): seq[Token] =
while not self.done():
self.start = self.current
self.scanToken()
# if self.tokens[^1].kind != TokenType.SEMICOLON:
# self.tokens.add(self.createToken(TOKENS[';']))
self.tokens.add(Token(kind: TokenType.EOF, lexeme: "EOF", line: self.line))
return self.tokens

View File

@ -22,7 +22,7 @@ import ../types/baseObject
type
CallFrame* = ref object # FIXME: Call frames are broken (end indexes are likely wrong)
CallFrame* = ref object
function*: ptr Function
ip*: int
slot*: int
@ -49,12 +49,8 @@ proc `[]`*(self: CallFrame, idx: int): ptr Obj =
proc `[]=`*(self: CallFrame, idx: int, val: ptr Obj) =
if idx < self.slot:
raise newException(IndexDefect, "CallFrame index out of range")
self.stack[idx + self.slot] = val
proc delete*(self: CallFrame, idx: int) =
if idx < self.slot:
raise newException(IndexDefect, "CallFrame index out of range")
self.stack.delete(idx + self.slot)

View File

@ -328,18 +328,18 @@ proc run(self: VM, repl: bool): InterpretResult =
{.computedgoto.} # See https://nim-lang.org/docs/manual.html#pragmas-computedgoto-pragma
instruction = frame.readByte()
opcode = OpCode(instruction)
## This offset dictates how the call frame behaves when converting
## relative frame indexes to absolute stack indexes, since the behavior
## in function local vs. global/scope-local scope is different
when DEBUG_TRACE_VM: # Insight inside the VM
stdout.write("Current VM stack status: [")
for v in self.stack:
for i, v in self.stack:
stdout.write(stringify(v))
stdout.write(", ")
if i < self.stack.high():
stdout.write(", ")
stdout.write("]\n")
stdout.write("Current global scope status: {")
for i, (k, v) in enumerate(self.globals.pairs()):
stdout.write("'")
stdout.write(k)
stdout.write("'")
stdout.write(": ")
stdout.write(stringify(v))
if i < self.globals.len() - 1:
@ -352,6 +352,13 @@ proc run(self: VM, repl: bool): InterpretResult =
stdout.write(&"function, '{frame.function.name.stringify()}'\n")
echo &"Current frame count: {self.frameCount}"
echo &"Current frame length: {frame.len}"
stdout.write("Current frame constants table: ")
stdout.write("[")
for i, e in frame.function.chunk.consts:
stdout.write(stringify(e))
if i < frame.function.chunk.consts.high():
stdout.write(", ")
stdout.write("]\n")
stdout.write("Current frame stack status: ")
stdout.write("[")
for i, e in frame.getView():

18
tests/japl/callchain.jpl Normal file
View File

@ -0,0 +1,18 @@
fun add2(x)
{
return x + 2;
}
fun sub2(x)
{
return x - 2;
}
fun mul2(x)
{
return x * 2;
}
print(add2(sub2(mul2(sub2(5)))));
//5-2=3
//3*2=6
//output:6

View File

@ -0,0 +1,42 @@
var x = 4;
var y = 5;
var z = 6;
if (x < y)
print("1");//output:1
else
print("2");
if (x == y)
print("3");//output:4
else
print("4");
if (x > y)
print("5");//output:6
else if (x < y)
print("6");
if (y >= 5)
print("7");//output:7
else
print("8");
if (z >= 5)
print("9");//output:9
else
print("10");
if (x <= 4)
print("11");//output:11
else
print("12");
if (2 <= y)
print("13");//output:13
else
print("14");
if (8 <= z)
print("15");
else
print("16");//output:16

View File

@ -0,0 +1,86 @@
var y = 0; //a global to keep track of state
//does not need closures for this to work yet
fun next(x) {
if (x == 10)
{
y = y + 1;
x = 0;
}
if (y == 10)
return -1;
return x+y+1;
}
var i = 0;
for (; i != -1; i = next(i))
print(i);
// before using next
//output:0
// y = 0
//output:1
//output:2
//output:3
//output:4
//output:5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 1
//output:2
//output:3
//output:4
//output:5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 2
//output:3
//output:4
//output:5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 3
//output:4
//output:5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 4
//output:5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 5
//output:6
//output:7
//output:8
//output:9
//output:10
// y = 6
//output:7
//output:8
//output:9
//output:10
// y = 7
//output:8
//output:9
//output:10
// y = 8
//output:9
//output:10
// y = 9
//output:10
// y = 10

4
tests/japl/hello.jpl Normal file
View File

@ -0,0 +1,4 @@
print("Hello, world.");
//output:Hello, world.
//output:

4
tests/japl/hellojapl.jpl Normal file
View File

@ -0,0 +1,4 @@
print("Hello, JAPL.");
//output:Hello, JAPL.
//output:

22
tests/japl/ifchain.jpl Normal file
View File

@ -0,0 +1,22 @@
fun printInt(x) {
if (x == 1)
print("one");
else if (x == 2)
print("two");
else if (x == 3)
print("three");
else if (x == 4)
print("four");
else if (x == 5)
print("five");
else if (x == 6)
print("six");
}
var x = 3;
printInt(x);//output:three
x = 5;
printInt(x);//output:five
x = 7;
printInt(x);
x = 1;
printInt(x);//output:one

22
tests/japl/is.jpl Normal file
View File

@ -0,0 +1,22 @@
var x = 4;
var y = x;
print(x is y);//output:false
print(x is x);//output:true
print(x is 4);//output:false
var z = true;
var u = true;
print(z is u);//output:true
print(z is x);//output:false
print(z is z);//output:true
var l = false;
print((not l) is z);//output:true
print(l is z);//output:false
print((l is z) is l);//output:true
var k;
print(k is nil);//output:true

5
tests/japl/nan.jpl Normal file
View File

@ -0,0 +1,5 @@
print((5/0)*0);
//output:nan
//output:

View File

@ -0,0 +1,4 @@
var a = 1; { var a = a; }
//output:A fatal error occurred while compiling '', line 1, at ';' -> cannot read local variable in its own initializer
//output:

View File

@ -0,0 +1,25 @@
{
var x = 5;
var y = x;
y = 6;
print(x);//output:5
}
var g = 7;
var p = g;
{
var k = g;
p = 3;
k = 9;
print(g);//output:7
}
print(g);//output:7
fun resetter(x) {
x = 7;
print(x);
}
var q = 5;
resetter(q);//output:7
print(q);//output:5

76
tests/japl/shadowing.jpl Normal file
View File

@ -0,0 +1,76 @@
//similar to vars.jpl, but more focused on shadowing
// simple shadowing
var x = 4;
{
var x = 5;
print(x);//output:5
}
print(x);//output:4
// type changing shadowing
var y = true;
{
var y = 2;
print(y);//output:2
}
print(y);//output:true
// no shadowing here
var z = 3;
{
z = true;
print(z);//output:true
}
print(z);//output:true
//in-function shadowing
fun shadow(x) {
//will be called once with the input 3
print(x);//output:3
{
var x = 4;
print(x);//output:4
}
print(x);//output:3
x = nil;
print(x);//output:nil
return x;
}
print(shadow(3));//output:nil
//shadowing functions
fun hello() {
print("hello");
}
hello();//output:hello
{
fun hello() {
print("hello in");
}
hello();//output:hello in
{
fun hello() {
print("hello inmost");
}
hello();//output:hello inmost
}
hello();//output:hello in
}
hello();//output:hello
//functions shadowing with type change
fun eat() {
print("nom nom nom");
}
eat();//output:nom nom nom
{
var eat = 4;
print(eat);//output:4
{{{{{
eat = 5;
}}}}} //multiple scopes haha
print(eat);//output:5
}
eat();//output:nom nom nom

12
tests/japl/strings.jpl Normal file
View File

@ -0,0 +1,12 @@
var left = "left";
var right = "right";
var directions = left + " " + right;
print(directions);//output:left right
var longstring = directions * 5;
print(longstring);//output:left rightleft rightleft rightleft rightleft right
left = left + " side";
print(left);//output:left side
right = "side: " + right;
print(right);//output:side: right

8
tests/japl/undefname.jpl Normal file
View File

@ -0,0 +1,8 @@
var a = b;
//output:Traceback (most recent call last):
//output: File '', line 1, in '<module>':
//output:ReferenceError: undefined name 'b'
//output:

View File

@ -0,0 +1,8 @@
var a = 2 + "hey";
//output:Traceback (most recent call last):
//output: File '', line 1, in '<module>':
//output:TypeError: unsupported binary operator '+' for objects of type 'integer' and 'string'
//output:

91
tests/maketest.nim Normal file
View File

@ -0,0 +1,91 @@
# Copyright 2020 Mattia Giambirtone
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Test creation tool, use mainly for exceptions
#
# Imports nim tests as well
import multibyte, os, strformat, times, re, terminal, strutils
const tempCodeFile = ".tempcode_drEHdZuwNYLqsQaMDMqeNRtmqoqXBXfnCfeqEcmcUYJToBVQkF.jpl"
const tempOutputFile = ".tempoutput.txt"
proc autoremove(path: string) =
if fileExists(path):
removeFile(path)
when isMainModule:
var testsDir = "tests" / "japl"
var japlExec = "src" / "japl"
var currentDir = getCurrentDir()
# Supports running from both the project root and the tests dir itself
if currentDir.lastPathPart() == "tests":
testsDir = "japl"
japlExec = ".." / japlExec
if not fileExists(japlExec):
echo "JAPL executable not found"
quit(1)
if not dirExists(testsDir):
echo "Tests dir not found"
quit(1)
echo "Please enter the JAPL code or specify a file containing it with file:<path>"
let response = stdin.readLine()
if response =~ re"^file:(.*)$":
let codepath = matches[0]
writeFile(tempCodeFile, readFile(codepath))
else:
writeFile(tempCodeFile, response)
let japlCode = readFile(tempCodeFile)
discard execShellCmd(&"{japlExec} {tempCodeFile} > {tempOutputFile} 2>&1")
var output: string
if fileExists(tempOutputFile):
output = readFile(tempOutputFile)
else:
echo "Temporary output file not detected, aborting"
quit(1)
autoremove(tempCodeFile)
autoremove(tempOutputFile)
echo "Got the following output:"
echo output
echo "Do you want to keep it as a test? [y/N]"
let keepResponse = ($stdin.readLine()).toLower()
let keep = keepResponse[0] == 'y'
if keep:
block saving:
while true:
echo "Please name the test (without the .jpl extension)"
let testname = stdin.readLine()
if testname == "":
echo "aborted"
break saving # I like to be explicit
let testpath = testsDir / testname & ".jpl"
echo &"Generating test at {testpath}"
var testContent = japlCode
for line in output.split('\n'):
var mline = line
mline = mline.replace(tempCodeFile, "")
testContent = testContent & "\n" & "//output:" & mline & "\n"
if fileExists(testpath):
echo "Test already exists"
else:
writeFile(testpath, testContent)
break saving
else:
echo "Aborting"

View File

@ -19,14 +19,29 @@
# - Assumes "japl" binary in ../src/japl built with all debugging off
# - Goes through all tests in (/tests/)
# - Runs all tests in (/tests/)japl/ and checks their output (marked by `//output:{output}`)
#
# Imports nim tests as well
import multibyte, os, strformat, times, re
import multibyte, os, strformat, times, re, terminal, strutils
const tempOutputFile = ".testoutput.txt"
const testResultsPath = "testresults.txt"
# Exceptions for tests that represent not-yet implemented behaviour
const exceptions = ["all.jpl"]
const exceptions = ["all.jpl", "for_with_function.jpl"]
# for_with_function.jpl probably contains an algorithmic error too
# TODO: fix that test
type LogLevel {.pure.} = enum
Debug, # always written to file only (large outputs, such as the entire output of the failing test or stacktrace)
Info, # important information about the progress of the test suite
Error, # failing tests (printed with red)
Stdout, # always printed to stdout only (for cli experience)
const echoedLogs = { LogLevel.Info, LogLevel.Error, LogLevel.Stdout }
const savedLogs = { LogLevel.Debug, LogLevel.Info, LogLevel.Error }
proc compileExpectedOutput(path: string): string =
@ -35,37 +50,52 @@ proc compileExpectedOutput(path: string): string =
result &= matches[0] & "\n"
proc deepComp(left, right: string): tuple[same: bool, place: int] =
proc deepComp(left, right: string, path: string): tuple[same: bool, place: int] =
var mleft, mright: string
result.same = true
if left.high() != right.high():
result.same = false
for i in countup(0, left.high()):
if left.replace(path, "").high() == right.replace(path, "").high():
mleft = left.replace(path, "")
mright = right.replace(path, "")
else:
result.same = false
else:
mleft = left
mright = right
for i in countup(0, mleft.high()):
result.place = i
if i > right.high():
if i > mright.high():
# already false because of the len check at the beginning
# already correct place because it's updated every i
return
if left[i] != right[i]:
if mleft[i] != mright[i]:
result.same = false
return
# Quick logging levels using procs
proc logWithLevel(level: LogLevel, file: File, msg: string) =
let msg = &"[{$level} - {$getTime()}] {msg}"
proc log(file: File, msg: string, toFile: bool = true) =
## Logs to stdout and to the log file unless
## toFile == false
if toFile:
file.writeLine(&"[LOG - {$getTime()}] {msg}")
echo &"[LOG - {$getTime()}] {msg}"
if level in savedLogs:
file.writeLine(msg)
if level in echoedLogs:
if level == LogLevel.Error:
setForegroundColor(fgRed)
echo msg
if level == LogLevel.Error:
setForegroundColor(fgDefault)
proc detail(file: File, msg: string) =
## Logs only to the log file
file.writeLine(&"[DETAIL - {$getTime()}] {msg}")
proc main(testsDir: string, japlExec: string, testResultsFile: File): tuple[numOfTests: int, successTests: int, failedTests: int, skippedTests: int] =
template detail(msg: string) =
logWithLevel(LogLevel.Debug, testResultsFile, msg)
template log(msg: string) =
logWithLevel(LogLevel.Info, testResultsFile, msg)
template error(msg: string) =
logWithLevel(LogLevel.Error, testResultsFile, msg)
var numOfTests = 0
var successTests = 0
var failedTests = 0
@ -73,61 +103,62 @@ proc main(testsDir: string, japlExec: string, testResultsFile: File): tuple[numO
try:
for file in walkDir(testsDir):
block singleTest:
for exc in exceptions:
if exc == file.path.extractFilename:
detail(testResultsFile, &"Skipping '{file.path}'")
numOfTests += 1
skippedTests += 1
break singleTest
if file.path.dirExists():
detail(testResultsFile, "Descending into '" & file.path & "'")
if file.path.extractFilename in exceptions:
detail(&"Skipping '{file.path}'")
numOfTests += 1
skippedTests += 1
break singleTest
elif file.path.dirExists():
detail(&"Descending into '" & file.path & "'")
var subTestResult = main(file.path, japlExec, testResultsFile)
numOfTests += subTestResult.numOfTests
successTests += subTestResult.successTests
failedTests += subTestResult.failedTests
skippedTests += subTestResult.skippedTests
break singleTest
detail(testResultsFile, &"Running test '{file.path}'")
if fileExists("testoutput.txt"):
removeFile("testoutput.txt") # in case this crashed
let retCode = execShellCmd(&"{japlExec} {file.path} >> testoutput.txt")
detail(&"Running test '{file.path}'")
if fileExists(tempOutputFile):
removeFile(tempOutputFile) # in case this crashed
let retCode = execShellCmd(&"{japlExec} {file.path} > {tempOutputFile} 2>&1")
numOfTests += 1
if retCode != 0:
failedTests += 1
log(testResultsFile, &"Test '{file.path}' has crashed!")
error(&"Test '{file.path}' has crashed!")
else:
successTests += 1
let expectedOutput = compileExpectedOutput(file.path).replace(re"(\n*)$", "")
let realOutputFile = open("testoutput.txt", fmRead)
let realOutputFile = open(tempOutputFile, fmRead)
let realOutput = realOutputFile.readAll().replace(re"([\n\r]*)$", "")
realOutputFile.close()
removeFile("testoutput.txt")
let comparison = deepComp(expectedOutput, realOutput)
removeFile(tempOutputFile)
let comparison = deepComp(expectedOutput, realOutput, file.path)
if comparison.same:
log(testResultsFile, &"Test '{file.path}' was successful")
successTests += 1
log(&"Test '{file.path}' was successful")
else:
detail(testResultsFile, &"Expected output:\n{expectedOutput}\n")
detail(testResultsFile, &"Received output:\n{realOutput}\n")
detail(testResultsFile, &"Mismatch at pos {comparison.place}")
if comparison.place > expectedOutput.high() or
comparison.place > realOutput.high():
detail(testResultsFile, &"Length mismatch")
failedTests += 1
detail(&"Expected output:\n{expectedOutput}\n")
detail(&"Received output:\n{realOutput}\n")
detail(&"Mismatch at pos {comparison.place}")
if comparison.place > expectedOutput.high() or comparison.place > realOutput.high():
detail(&"Length mismatch")
else:
detail(testResultsFile, &"Expected is '{expectedOutput[comparison.place]}' while received '{realOutput[comparison.place]}'")
log(testResultsFile, &"Test '{file.path}' failed")
detail(&"Expected is '{expectedOutput[comparison.place]}' while received '{realOutput[comparison.place]}'")
error(&"Test '{file.path}' failed")
result = (numOfTests: numOfTests, successTests: successTests, failedTests: failedTests, skippedTests: skippedTests)
except IOError:
stderr.write(&"Fatal IO error encountered while running tests -> {getCurrentExceptionMsg()}")
when isMainModule:
let testResultsFile = open("testresults.txt", fmWrite)
log(testResultsFile, "Running Nim tests")
let testResultsFile = open(testResultsPath, fmWrite)
template log (msg: string) =
logWithLevel(LogLevel.Info, testResultsFile, msg)
log("Running Nim tests")
# Nim tests
detail(testResultsFile, "Running testMultiByte")
logWithLevel(LogLevel.Debug, testResultsFile, "Running testMultiByte")
testMultiByte()
# JAPL tests
log(testResultsFile, "Running JAPL tests")
log("Running JAPL tests")
var testsDir = "tests" / "japl"
var japlExec = "src" / "japl"
var currentDir = getCurrentDir()
@ -135,16 +166,16 @@ when isMainModule:
if currentDir.lastPathPart() == "tests":
testsDir = "japl"
japlExec = ".." / japlExec
log(testResultsFile, &"Looking for JAPL tests in {testsDir}")
log(testResultsFile, &"Looking for JAPL executable at {japlExec}")
log(&"Looking for JAPL tests in {testsDir}")
log(&"Looking for JAPL executable at {japlExec}")
if not fileExists(japlExec):
log(testResultsFile, "JAPL executable not found")
log("JAPL executable not found")
quit(1)
if not dirExists(testsDir):
log(testResultsFile, "Tests dir not found")
log("Tests dir not found")
quit(1)
let testResult = main(testsDir, japlExec, testResultsFile)
log(testResultsFile, &"Found {testResult.numOfTests} tests: {testResult.successTests} were successful, {testResult.failedTests} failed and {testResult.skippedTests} were skipped.")
log(testResultsFile, "Check 'testresults.txt' for details", toFile=false)
log(&"Found {testResult.numOfTests} tests: {testResult.successTests} were successful, {testResult.failedTests} failed and {testResult.skippedTests} were skipped.")
logWithLevel(LogLevel.Stdout, testResultsFile, "Check 'testresults.txt' for details")
testResultsfile.close()