- Can make the event buffer larger or smaller
- UI shows the current event index / size, so you know how fast it's
filling up
- Can save compressed, 10x reduction in filesize and Windows 11 explorer
actually supports ZSTD natively now so this isn't inconvenient at all
> An example of almost 1 million events. Results in a 4mb file.
This PR does two main things:
1. Work through the main low-hanging fruit issues in the formatter
keeping it from feeling mature and usable
2. Iterate and prove that point by formatting all of the Jak 1 code
base. **This has removed around 100K lines in total.**
- The decompiler will now format it's results for jak 1 to keep things
from drifting back to where they were. This is controlled by a new
config flag `format_code`.
How am I confident this hasn't broken anything?:
- I compiled the entire project and stored it's `out/jak1/obj` files
separately
- I then recompiled the project after formatting and wrote a script that
md5's each file and compares it (`compare-compilation-outputs.py`
- The results (eventually) were the same:
> This proves that the only difference before and after is non-critical
whitespace for all code/macros that is actually in use.
I'm still aware of improvements that could be made to the formatter, as
well as general optimization of it's performance. But in general these
are for rare or non-critical situations in my opinion and I'll work
through them before doing Jak 2. The vast majority looks great and is
working properly at this point. Those known issues are the following if
you are curious:
- `speech`
- `ambient`
- `water-h`
- `vol-h`
- `generic-obs`
- `carry-h`
- `pilot-h`
- `board-h`
- `gun-h`
- `flut-h`
- `indax-h`
- `lightjak-h`
- `darkjak-h`
- `target-util`
- `history`
- `collide-reaction-target`
- `logic-target`
- `sidekick`
- `projectile`
- `voicebox`
- `ragdoll-edit`
- most of `ragdoll` (not added to gsrc yet)
- `curves`
- `find-nearest`
- `lightjak-wings`
- `target-handler`
- `target-anim`
- `target`
- `target2`
- `target-swim`
- `target-lightjak`
- `target-invisible`
- `target-death`
- `target-gun`
- `gun-util`
- `board-util`
- `target-board`
- `board-states`
- `mech-h`
- `vol`
- `vent`
- `viewer`
- `gem-pool`
- `collectables`
- `crates`
- `secrets-menu`
Additionally:
- Detection of non-virtual state inheritance
- Added a config file that allows overriding the process stack size set
by `stack-size-set!` calls
- Fix for integer multiplication with `r0`
- Fixed detection for the following macros:
- `static-attack-info`
- `defpart` and `defpartgroup` (probably still needs adjustments, uses
Jak 2 implementation at the moment)
- `sound-play` (Jak 3 seems to always call `sound-play-by-name` with a
`sound-group` of 0, so the macro has been temporarily defaulted to use
that)
One somewhat significant change made here that should be noted is that
the return type of `process::init-from-entity!` was changed to `object`.
I've been thinking about this for a while, since it looks a bit nicer
without the `(none)` at the end and I have recently encountered init
methods that early return `0`.
In #3300, `per_object` in `ObjectFileDB::ir2_analyze_all_types()` was
changed to an `unordered_map`, which messed with the object file order
in `all-types`. This fixes the issue by going through each object file
in order and then looking up the type info from the map instead of
iterating through the map.
This also removes the method IDs from the old method comments and adds a
stub for the `new` method if there is an old method comment for it, in
order to stay consistent with the rest of the method list.
This sets up the C Kernel for Jak 3, and makes it possible to build and
load code built with `goalc --jak3`.
There's not too much interesting here, other than they switched to a
system where symbol IDs (unique numbers less than 2^14) are generated at
compile time, and those get included in the object file itself.
This is kind of annoying, since it means all tools that produce a GOAL
object file need to work together to assign unique symbol IDs. And since
the symbol IDs can't conflict, and are only a number between 0 and 2^14,
you can't just hash and hope for no collisions.
We work around this by ignoring the IDs and re-assigning our own. I
think this is very similar to what the C Kernel did on early builds of
Jak 3 which supported loading old format level files, which didn't have
the IDs included.
As far as I can tell, this shouldn't cause any problems. It defeats all
of their fancy tricks to save memory by not storing the symbol string,
but we don't care.
This sets out the bones of a Jak 3 build, many things are stubbed out,
guessed, or copied from Jak 2 but it should at least be good enough to:
run `task set-game-jak3`
launch the repl
run builds from the repl
build outputs themselves are untested but the build itself runs without
errors
---------
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
The logger used in `goalc` tries to print an already-formatted string
`message` using `fmt::print(message);` Usually this doesn't cause
problems, but if you try to print, for example, an exception that has
special characters (notably `{`) it will try to do another round of
formatting/replacements, despite not having any args to replace with,
which ends up throwing another exception. This is why errors when
parsing custom level JSON cause the REPL to exit.
I've hopefully identified all the various instances of this across the
codebase
Adds sprite distort, fixes buggy sprite rendering in progress, adds
scissoring support (used in various scrolling menus) and a very basic
implementation of `blit-displays`. This is enough to make the fade
effect in the progress menu work, along with all the menus working
properly without needing to use the REPL. This does not make screen
flipping and the filter when failing a mission work.
Added support in the decompiler for detecting `dma-buffer-add-gs-set`
and `dma-buffer-add-gs-set-flusha` and updated all of the Jak 2 code to
use it. Readability improved!
Fixes decompiler issue with `with-dma-buffer-add-bucket` not inlining
forms which broke syntax. Fixes store error warnings showing up for
non-existent stores, there is now a dedicated pass for this at the end.
I started work on making `BITBLTBUF` stuff work in the DirectRenderer,
but stopped for now because it wasn't strictly necessary. It will still
assert like before.
When working with mips2c recently, I noticed adding the
`defmethod-mips2c` or `def-mips2c` code was a manual step. This is a bit
tedious to have to go and do yourself, but more importantly you have to
manually go and find the right spot in the source file else you might be
declaring it too early or too late.
This will automatically output the declaration for methods, and a
half-finished comment for the functions. I wasn't able to fully output
the function one because it seems the signature info from `all-types`
doesn't make it all the way through -- but maybe I'm wrong or this is an
easy fix?
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular
The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it
https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.
I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.
We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.
I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.
Closes#1358
* all-types: improve all-types generation
* all-types: re-generate all-types
* tests: remove the test reporting feature
the format indeed doesn't work, and all current actions require too many permissions for forked PRs.
I'll make my own eventually that works properly (use the new markdown feature)
* all-types: put the states in the method table instead
* all-types: replace all `*time*...uint64` fields with `time-frame` type
* all-types: address feedback
* lsp: json-rpc example is working, a decent place to start...
* lsp: vendor library
* lsp: cleanup and time to get started
* lsp: commit what i got so far
* lsp: example `initialize` payload
* lsp: switch to `stdio`
* stash
* modularize the lsp implementation
* lsp: implement first actual LSP feature - function names in outline
* lsp: produce document diagnostics
* lsp: remove unused third-party lib
* lsp: support hovering MIPS instructions in IR files
* lsp: basic go-to all-types definition
* stash
* lsp: cleanup code, just need to add it to the release artifacts
* fix some project configuration
* fix linux build
* lsp: add lsp to PR artifacts and release assets
* lsp: address feedback
* extractor: refactor and cleanup for multi-game support
* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default
* extractor: finally working with unicode
* unicode: fix unicode cli args on windows in all `main` functions
* Remove assets folder, use more std::filesystem
* windows fix
* another one for windows
* another one
* better system for different folders
* rm debugging stuff
* let extractor override everything
* dont revert jak1 change
* some jp support to fix some errors in the original game
* music fade toggle
* recognize `process-new` macros!!
* strip casts in this macro
* rename macro
* fix cast typecheck
* update source 1
* detect kernel stack case
* less boilerplate
* `manipy-spawn` special case
* pretty printer improvements
* revert dumb thing from earlier
* use shell detection on `send-event`
* fix some events
* remove unused argument
* detect `static-attack-info` and add `CondNoElse` to shell detect
* better `attack-info` detect
* support `process-spawn` in multi-lets
* detect `rand-float-gen` pt 1
* detect as return value
* detect in `countdown` and `dotimes`
* oops this wasnt working
* fancier `send-event`s
* clang
* update source!!
* fix tests
* fine jeez
* uh okay
* fix some accidental regressions
* fix more regressions
* regression fixes
* fix big bug...
* extra safety!
* update refs
* [decompiler] read and process art groups
* finish decompiler art group selection & detect in `ja-group?`
* make art stuff work on offline tests!
* [decompiler] detect `ja-group!` (primitive)
* corrections.
* more
* use new feature on skel groups!
* find `loop!` as well
* fully fledged `ja` macro & decomp + `loop` detect
* fancy fixed point printing!
* update source
* `:num! max` (i knew i should've done this)
* Update jak1_ntsc_black_label.jsonc
* hi imports
* make compiling the game work
* fix `defskelgroup`
* clang
* update refs
* fix chan
* fix seek and finalboss
* fix tests
* delete unused function
* track let rewrite stats
* reorder `rewrite_let`
* Update .gitattributes
* fix bug with `:num! max`
* Update robotboss-part.gc
* Update goal-lib.gc
* document `ja`
* get rid of pc fixes thing
* use std::abs
* [decompiler] `defpart` and `defpartgroup`
* support sounds in part defs natively
* make `meters_to_string`
* update refs
* clang
* make macros in pair only work inside `(unquote ...`
* update source
* update finish
* fix
* begin work
* work
* working objs
* exporting
* it works
* before some time of day fixes
* add time of day interp and also fix zbuffer
* some small blending fixes
* improve randomess
* clean up extraction and missing blend mode
* culling, time of day, more level fixes
* more cleanup
* cleanup memory usage
* windows fix
* implement `defskelgroup` in decompiler
0 lines removed!
* clang
* fix matcher code
* it was fine actually
* update all files (pt 1)
* update all files
* remaining fixes!
* null deref
* use METER_LENGTH
* move max lod warning
* clang
* math error
* fix macros too
* fix bug in a print
* decomp: mostly finish `cam-master`
* decomp/scripts: lots of work in cam-states
* stash
* Merge remote-tracking branch 'water111/master' into decomp/camera-master
Updated submodule third-party/googletest
* decompiler: Add support for non power of 2 offsets for inline arr access
* decomp: mostly finish `cam-states` need to fix a macro issue
* blocked: `cam-master` decompiler crash when adding casts
* decomp: finish `cam-states-dbg`
* decomp: mostly finish `pov-camera` with the exception of joint-related code
* decomp: `cam-debug` finished decompiling, no way does this compile yet though
* decomp: considerable work done in `cam-layout`
* decomp: `cam-layout` almost done!
* decomp: `pov-camera` finished, TC tests will fail for now
* decomp: working on resolving issues
* decomp: cam-layout decompiling
* fixing more issues in cam-master...one event handler remains
* skip problematic function in `cam-master` for now
* gsrc: update res macros
* decomp: finish `cam-states`
* decomp: giving up on `cam-debug`
* tests: allow skipping state handlers in ref tests
* decomp: working through cam-layout bugs
* decomp: allow for shifting non-integers
* decomp: finalize `cam-layout` and `cam-master`
* decomp: finalize `cam-states`
* cleanup: bi-annual formatting of the casting files
* formatting
* address feedback - leave the float labels alone for now
* address feedback
* linting/formatting
* update gsrc and ref tests
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
