This automatically generates documentation from goal_src docstrings,
think doxygen/java-docs/rust docs/etc. It mostly supports everything
already, but here are the following things that aren't yet complete:
- file descriptions
- high-level documentation to go along with this (think pure markdown
docs describing overall systems that would be co-located in goal_src for
organizational purposes)
- enums
- states
- std-lib functions (all have empty strings right now for docs anyway)
The job of the new `gen-docs` function is solely to generate a bunch of
JSON data which should give you everything you need to generate some
decent documentation (outputting markdown/html/pdf/etc). It is not it's
responsibility to do that nice formatting -- this is by design to
intentionally delegate that responsibility elsewhere. Side-note, this is
about 12-15MB of minified json for jak 2 so far :)
In our normal "goal_src has changed" action -- we will generate this
data, and the website can download it -- use the information to generate
the documentation at build time -- and it will be included in the site.
Likewise, if we wanted to include docs along with releases for offline
viewing, we could do so in a similar fashion (just write a formatting
script to generate said documentation).
Lastly this work somewhat paves the way for doing more interesting
things in the LSP like:
- whats the docstring for this symbol?
- autocompleting function arguments
- type checking function arguments
- where is this symbol defined?
- etc
Fixes#2215
- lets you split up your `startup.gc` file into two sections
- one that runs on initial startup / reloads
- the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do
Co-authored-by: water <awaterford111445@gmail.com>
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular
The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it
https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.
As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)
This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
"numConnectToTargetAttempts": 1,
"jak2": {
"asmFileSearchDirs": [
"goal_src/jak2"
]
}
}
```
This also provides a way to make game-specific configurations for the
REPL fairly easily.
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here
https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.
If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
"-workbench.action.quickOpen",
"-workbench.action.quickOpenView"
]
```
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.
I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.
We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.
I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.
Closes#1358
And everything else needed for them!
A couple functions are bad currently.
- fixes#1929 - untested on linux
- fixes#1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
Fixes https://github.com/open-goal/jak-project/issues/1821 by adding a
special case for `new` method calls where the argument with type
`symbol` is actually an address to uninitialized structure on the stack.
Fixes https://github.com/open-goal/jak-project/issues/1849 (or at least
the cause of the issue Vaser gave in chat, and one random one I found in
`debug-sphere`)
Fixes https://github.com/open-goal/jak-project/issues/1853
Fixes https://github.com/open-goal/jak-project/issues/1857 by moving the
cast into the cond if the body is a single form and the destination type
is a bitfield/enum which is likely to work well. Seems to work on the
examples we could find in jak 1 and jak 2.
Also fixes an issue with casts on the result of `handle->process` (a
common place to use casts)
the output of process->handle is a plain process. Most of the time, you
end up casting this to a more specific. If you add a cast on every use
of the variable, the decompiler will decide to change the type of that
variable to the more specific type, and this breaks the handle cast.
so previously it was impossible to get code like
```
(let* ((s2-0 (the-as swingpole (handle->process (-> self control hack))))
(gp-0 (-> s2-0 dir))
)
```
But now it will work
* lsp: json-rpc example is working, a decent place to start...
* lsp: vendor library
* lsp: cleanup and time to get started
* lsp: commit what i got so far
* lsp: example `initialize` payload
* lsp: switch to `stdio`
* stash
* modularize the lsp implementation
* lsp: implement first actual LSP feature - function names in outline
* lsp: produce document diagnostics
* lsp: remove unused third-party lib
* lsp: support hovering MIPS instructions in IR files
* lsp: basic go-to all-types definition
* stash
* lsp: cleanup code, just need to add it to the release artifacts
* fix some project configuration
* fix linux build
* lsp: add lsp to PR artifacts and release assets
* lsp: address feedback
* fix utf-8 handling around env-vars
* fix file opening errors related to unicode
* add uncaught exception handler in `gk` to ensure something is logged
* gracefully fail if window icon cant be loaded and work with unicode
* linux fix and add changes to vendor file
* extractor: refactor and cleanup for multi-game support
* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default
* extractor: finally working with unicode
* unicode: fix unicode cli args on windows in all `main` functions
* fix repl-history saving
* subtitles: fix `\"` edge-case
* subtitles: add almost every remaining (current) cutscene
* merge: master
* subtitles: fix some crashes and disable `Remove Line` as it is broken
* some jp support to fix some errors in the original game
* music fade toggle
* recognize `process-new` macros!!
* strip casts in this macro
* rename macro
* fix cast typecheck
* update source 1
* detect kernel stack case
* less boilerplate
* `manipy-spawn` special case
* pretty printer improvements
* revert dumb thing from earlier
* use shell detection on `send-event`
* fix some events
* remove unused argument
* detect `static-attack-info` and add `CondNoElse` to shell detect
* better `attack-info` detect
* support `process-spawn` in multi-lets
* detect `rand-float-gen` pt 1
* detect as return value
* detect in `countdown` and `dotimes`
* oops this wasnt working
* fancier `send-event`s
* clang
* update source!!
* fix tests
* fine jeez
* uh okay
* fix some accidental regressions
* fix more regressions
* regression fixes
* fix big bug...
* extra safety!
* update refs
* [decompiler] read and process art groups
* finish decompiler art group selection & detect in `ja-group?`
* make art stuff work on offline tests!
* [decompiler] detect `ja-group!` (primitive)
* corrections.
* more
* use new feature on skel groups!
* find `loop!` as well
* fully fledged `ja` macro & decomp + `loop` detect
* fancy fixed point printing!
* update source
* `:num! max` (i knew i should've done this)
* Update jak1_ntsc_black_label.jsonc
* hi imports
* make compiling the game work
* fix `defskelgroup`
* clang
* update refs
* fix chan
* fix seek and finalboss
* fix tests
* delete unused function
* track let rewrite stats
* reorder `rewrite_let`
* Update .gitattributes
* fix bug with `:num! max`
* Update robotboss-part.gc
* Update goal-lib.gc
* document `ja`
* get rid of pc fixes thing
* use std::abs
* goalc: cleanup goalc's main method and add nrepl listener socket
* deps: add standalone ASIO for sockets
* lint: formatting
* common: make a common interface for creating a server socket
* goalc: setup new repl server
* deps: remove asio
* goalc: debug issues, nrepl is working again
* git: rename files
* attempt to fix linux function call
* test
* scripts: make the error message even more obvious....
* goalc: make suggested changes, still can't reconnect properly
* game: pull out single-client logic from XSocketServer
* nrepl: supports multiple clients and disconnection/reconnects
* goalc: some minor fixes for tests
* goalc: save repl history when the compiler reloads
* common: add include for linux networking
* a few small changes to fix tests
* is it the assert?
* change thread start order and add a print to an assert
Co-authored-by: water <awaterford111445@gmail.com>
* fix typo
* more typo
* shorten discord rpc text
* allow expanding enums after the fact (untested)
* make `game_text` work similar to subtitles
* update progress decomp
* update some types + `do-not-decompile` in bitfield
* fixes and fall back to original progress code
* update `progress` decomp with new enums
* update config files
* fix enums and debug menu
* always allocate (but not use) a lot of particles
* small rework to display mode options
* revert resolution/aspect-ratio symbol mess
* begin the override stuff
* make `progress-draw` more readable
* more fixes
* codacy good boy points
* first step overriding code
* finish progress overrides, game options menu fully functional!
* minor fixes
* Update game.gp
* Update sparticle-launcher.gc
* clang
* change camera controls text
* oops
* some cleanup
* derp
* nice job
* implement menu scrolling lol
* make scrollable menus less cramped, fix arrows
* make some carousell things i guess
* add msaa carousell to test
* oops
* Update progress-pc.gc
* make `pc-get-screen-size` (untested)
* resolution menu
* input fixes
* return when selecting resolution
* scroll fixes
* Update progress-pc.gc
* add "fit to screen" button
* bug
* complete resolutions menu
* aspect ratio menu
* subtitles language
* subtitle speaker
* final adjustments
* ref test
* fix tests
* fix ref!
* reduce redundancy a bit
* fix mem leaks?
* save settings on progress exit
* fix init reorder
* remove unused code
* rename goal project-like files to the project extension
* sha display toggle
* aspect ratio settings fixes
* dont store text db's in compiler
* properly save+load native aspect stuff
* [goalc] macro expansion in integer constants
* working
* didn't break it yet
* support conditional compilation
* fix up some more small bugs
* fix duplicate evaluation of bitfield definitions
* paranoid
* first attempt
* fix
* zip to tar
* windows
* try again, std::filesystem sucks
* std::filesystem is still garbage
* std::filesystem is terrible
* std::filesystem continues to waste my time
* again
* neadsflaldksal;df
* delete unused shaders
* hide some options in debug menu
* change fullscreen logic a bit
* add "all actors" toggle
* borderless fix and fix alpha in direct renderer untextured (do we need a separate shader for that?)
* fix fuel cell orbit icons in widescreen
* fix `curve` types
* refs
* fix levitator task...
* fix some task stuff
* update font code a bit (temp)
* cmake, third-party and visual studio overhaul
* Update .gitmodules
* update modules
* clone repos
* fix encoding in zydis
* where did these come from
* try again
* add submodule
* Update 11zip
* Update 11zip
* Update 11zip
* delete
* try again
* clang
* update compiler flags
* delete 11zip. go away.
* Create memory-dump-p2s.py
* properly
* fix minimum architecture c++ compiler flags
* fix zydis
* oops
* Update all-types.gc
* fix clang-cl tests
* make "all actors" work better, entity debug qol
* update game-text conversion code to be more modularized
* Create vendor.txt
* fix typos and minor things
* update refs
* clang
* Attempt to add clang-cl support to vs2019 and CI
* vs2022 + clang-cl
* srsly? fix clang build
* Update launch.vs.json
* extend windows CI timer
* [decompiler] `defpart` and `defpartgroup`
* support sounds in part defs natively
* make `meters_to_string`
* update refs
* clang
* make macros in pair only work inside `(unquote ...`
* update source
* update finish
* fix
* update progress code and fix our frame "scissoring"
* aspect ratio hacks
* expand progress tint for widescreen
* give up and comment things properly
* properly undo hacks
