The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
Almost done:
- `target-handler` (`(none)` event handler casts and CFG error)
- `target2` (`(none)` event handler casts)
- `powerups` (`cloud-track` does some weird stuff with `handle`s)
- `gun-states` (CFG error)
Some progress in:
- `water-flow`
Additionally:
- Clean up the two year old Jak 3 config file and add a config skeleton
(disassembling seems to not have worked, but I was able to dump obj
files and the `all_scripts` file)
- Fix automatic skelgroup detection and `defskelgroup` macro for Jak 2
(closes#1950)
- When a function decompiles without any major errors, a warning is
generated with the op id for each unresolved load and store that will
likely fail to compile (closes#1933)
This PR does a few main things:
- finish decompiling the progress related code
- implemented changes necessary to load the text files end-to-end
- japanese/korean character encodings were not added
- finish more camera code, which is required to spawn the progress menu
/ init the default language settings needed for text
- initialized the camera as well
Still havn't opened the menu as there are a lot of checks around
`*target*` which I havn't yet gone through and attempted to comment out.
A few issues:
- lwidea's fr3 is getting loaded and unloaded all the time
- the debug line drawing clipping is wrong (doesn't seem wrong in pcsx2,
so I think this is on us)
- nothing actually using vis data yet
- at a large distance, our view frustum culling seems slightly too
aggressive (might be that viewport scissoring is wrong)
- in the city, things seem darker as you move away. unclear how this is
happening (fog?)
And everything else needed for them!
A couple functions are bad currently.
- fixes#1929 - untested on linux
- fixes#1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
A small quality of life increase that is more impactful since jak 2 has
double the file count.
I often use the offline-tests to find compiler errors / automatically
resolve them when i am finalizing a file. As more and more files are
completed this becomes increasingly more inefficient. When I know that
only 1 file needs to be decompiled / compared / compiled, I'd prefer to
have a feature like this.
- decompile `subdivide`, `wind-work`, `tie-work`, `bsp`, `focus`
- support `ppacb` in compiler
- don't assert when bitfield stuff fails due to constant propgataion
weirdness
- finish up history
- div/mod unsigned assert fix in decompiler
- empty assert fix in decompiler for failed `add` type prop
- make jak 1 performance counters "work" (just measure time)
- fix cast/typos on pcgtb/vftoi15
I wasn't able to 100% complete `debug` due to a bunch of level boundary
debug stuff I couldn't figure out. But, I added a ref test and
documented/copied over everything else into goal_src.
Most of the functions that existed in Jak 1 are identical, others got an
extra param or 2 and some now make copies of arguments instead of
modifying them. There's a bunch of new functions, including all of the
debug functions that used to be in `level-boundary` are now in `debug`.
At the very end of `debug` there's also some weird asm functions
checking for some EE memory controller bug (not sure what's up with
that)?
* sparticle-launcher
* d/jak2: large amount of `sparticle-launcher` done
* d/jak2: finish the majority of `sparticle`
* decomp: improve format code ignoring
* d/jak2: make bits unique in `sp-cpuinfo-flag`
* d/jak1: revert config change
* d/jak2: finish `cylinder`
* d/jak2: mostly finish `history`
* d/jak2: finish a bit of `ripple` and `sync-info`
* d/jak2: finish `font-data` and chip away at some other files
* d/jak2: most of `text` done, a bit of `script` - symbol arr issue in `level-info`
* d/jak2: finish `mech-h`
* lint: format
* tests: fix typeconsistency
* d/jak2: address feedback
* fix utf-8 handling around env-vars
* fix file opening errors related to unicode
* add uncaught exception handler in `gk` to ensure something is logged
* gracefully fail if window icon cant be loaded and work with unicode
* linux fix and add changes to vendor file
* git: ignore vs build dir
* cmake: ditch `clang-cl` on windows in favor of actual `clang`
* build: suppress a significant number of warnings
* build: adjust workflows and vendor nasm
* docs: update docs to remove `clang-cl` mentions
* tests: move jak1 reference tests into their own folder
* tests: update offline tests to support multiple games
* tests: some additional fixes and multi-game handling
* tests: update reference tests
* extractor: refactor and cleanup for multi-game support
* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default
* extractor: finally working with unicode
* unicode: fix unicode cli args on windows in all `main` functions
* Remove assets folder, use more std::filesystem
* windows fix
* another one for windows
* another one
* better system for different folders
* rm debugging stuff
* let extractor override everything
* dont revert jak1 change
* PAL dumps files
* alrighty then
* make PAL warning slightly more specific
* PAL patches for `title-obs`
* Update all-types.gc
* PAL patch `beach-obs`
* `process-taskable` PAL patch
* `ambient` PAL patch
* `yakow` PAL patch
* `village-obs` PAL patch
* `sparticle-launcher` patch
* `swamp-obs` PAL patch
* `sequence-a-village1` PAL patch
* typo
* errors
* `powerups` PAL patch
* `ogreboss` PAL patch
* jak 1 v2 encoding
* `load-boundary` PAL patch
* `flying-lurker` PAL patch
* `mayor` PAL patch
* update game encoding to PAL (v2) encoding
* `cam-debug` and `cam-update` PAL patch
* `fisher` PAL patch
* `target` PAL patch
* `target2` PAL patch and fix text compiling
* `target-death` PAL patch
* `target-racer-h` PAL patch
* `logic-target` PAL patch
* `main` PAL patch
* `snow-flutflut-obs` PAL patch
* `rolling-obs` PAL patch
* `gsound` PAL patch
* update refs
* `progress` and `progress-draw` PAL patches
* clang
* wrong.
* complain
* clang
* fix test
* fix blurry jp text
* fix weird interrupt lag from setting window size
* patch more text lines, special handling for credits
* Update FontUtils.cpp
* Add xdelta3 and file patching interface
* add window lock toggle and update settings ver
* better particle hacks
* add PAL support to extractor
* Fix credits
* also NTSC-J support
* make xdelta3 a separate library
* address feedback
Co-authored-by: water <awaterford111445@gmail.com>
* update refs
* [decompiler] read and process art groups
* finish decompiler art group selection & detect in `ja-group?`
* make art stuff work on offline tests!
* [decompiler] detect `ja-group!` (primitive)
* corrections.
* more
* use new feature on skel groups!
* find `loop!` as well
* fully fledged `ja` macro & decomp + `loop` detect
* fancy fixed point printing!
* update source
* `:num! max` (i knew i should've done this)
* Update jak1_ntsc_black_label.jsonc
* hi imports
* make compiling the game work
* fix `defskelgroup`
* clang
* update refs
* fix chan
* fix seek and finalboss
* fix tests
* delete unused function
* track let rewrite stats
* reorder `rewrite_let`
* Update .gitattributes
* fix bug with `:num! max`
* Update robotboss-part.gc
* Update goal-lib.gc
* document `ja`
* get rid of pc fixes thing
* use std::abs
* first attempt
* fix
* zip to tar
* windows
* try again, std::filesystem sucks
* std::filesystem is still garbage
* std::filesystem is terrible
* std::filesystem continues to waste my time
* again
* neadsflaldksal;df
* begin generic merc
* more generic stuff
* generic dma generation not crashing, at least until high-speed-reject
* c
* unpacker for generic
* starting to work
* small fixes
* more fixes
* more progress
* cleanup
* fix tests
* no custom imgui assert on windows
* ci: switch to codacy for coverage
* docs: update badges
* decomp: allow overriding config flags via CLI
* cleanup: top level file cleanup
* docs: big README overhaul
Attempt to close#1128 and #1086
* decomp: attempt to detect if `iso_data` is missing or wrongly extracted
* game: switch to `fpng` for screenshots, allow for compression
closes#1035
* game: switch vsync control to a checkbox
* lint: format cpp files
* lint: format json files
* docs/scripts: organize taskfile
* delete unused shaders
* hide some options in debug menu
* change fullscreen logic a bit
* add "all actors" toggle
* borderless fix and fix alpha in direct renderer untextured (do we need a separate shader for that?)
* fix fuel cell orbit icons in widescreen
* fix `curve` types
* refs
* fix levitator task...
* fix some task stuff
* update font code a bit (temp)
* cmake, third-party and visual studio overhaul
* Update .gitmodules
* update modules
* clone repos
* fix encoding in zydis
* where did these come from
* try again
* add submodule
* Update 11zip
* Update 11zip
* Update 11zip
* delete
* try again
* clang
* update compiler flags
* delete 11zip. go away.
* Create memory-dump-p2s.py
* properly
* fix minimum architecture c++ compiler flags
* fix zydis
* oops
* Update all-types.gc
* fix clang-cl tests
* make "all actors" work better, entity debug qol
* update game-text conversion code to be more modularized
* Create vendor.txt
* fix typos and minor things
* update refs
* clang
* Attempt to add clang-cl support to vs2019 and CI
* vs2022 + clang-cl
* srsly? fix clang build
* Update launch.vs.json
* extend windows CI timer
* decomp: Add texture-upload to ref tests
* maybe 50% done?
* 5 functions to go!
* decomp: stuck in `navigate`
* work-around fp issue
* some cleanup and label casts
* working on supporting asm instructions -- this is currently WRONG
* support ASM operations
* fixes for asm op support
* decomp: finish the vast majority of `navigate`
* format
* update test though i think this suggests a regression!
* decomp: cleanup some more of navigate
* decomp: finish `rolling-lightning-mole`
* revert `r0` handling for `pcpyud` and `pextuw`
* update ref tests
* lint
* fix a failing test
* help
* navigate mostly works now, with some potential bugs
* remove my debugging logs
* update ref tests
* review feedback cleanup
* these are all likely fine
* can't get the crab to chase me anymore
* the crab is back
* cleanup: cleanup all-types and all state definitions
* these are fine
* these are fine
* these are fine
* everything is fine
* gsrc updated
* remove define-externs in gsrc
* decomp: finish `target-racer`
target-racing-turn-anim was the `cond` issue here
* decomp: finish `target-tube`
target-tube-turn-anim was the issue here
* temp
* temp
* wip
* more progress on the instance asm
* first half of tie extraction, up to dma lists
* more tie extraction
* first part figured out maybe
* bp1 loop seems to work, bp2 loop does not
* bp1 and bp2 appear working. sadly ip is needed
* ip1 outline, not working ip2
* just kidding, ip2 seems to work
* extraction seems to work
* basic rendering working
* tie fixes
* performance optimization of tie renderer
* hook up tie to engine
* fix more bugs
* cleanup and perf improvements
* fix tests
* ref tests
* mm256i for gcc
* CLANG
* windows
* more compile fixes
* fix fast time of day
* small fixes
* fix after merge
* clang
* stash
* more
* and more
* like 80% done
* trying to update things
* refreshed work on generic-obs
* blocked: down to only decompiler issues!
* decomp: finish `generic-obs` skip over the failing functions for now
* `pair` -> `object`s and fix 2/4 of the functions i had skipped
* address feedback
* decomp: mostly finish `cam-master`
* decomp/scripts: lots of work in cam-states
* stash
* Merge remote-tracking branch 'water111/master' into decomp/camera-master
Updated submodule third-party/googletest
* decompiler: Add support for non power of 2 offsets for inline arr access
* decomp: mostly finish `cam-states` need to fix a macro issue
* blocked: `cam-master` decompiler crash when adding casts
* decomp: finish `cam-states-dbg`
* decomp: mostly finish `pov-camera` with the exception of joint-related code
* decomp: `cam-debug` finished decompiling, no way does this compile yet though
* decomp: considerable work done in `cam-layout`
* decomp: `cam-layout` almost done!
* decomp: `pov-camera` finished, TC tests will fail for now
* decomp: working on resolving issues
* decomp: cam-layout decompiling
* fixing more issues in cam-master...one event handler remains
* skip problematic function in `cam-master` for now
* gsrc: update res macros
* decomp: finish `cam-states`
* decomp: giving up on `cam-debug`
* tests: allow skipping state handlers in ref tests
* decomp: working through cam-layout bugs
* decomp: allow for shifting non-integers
* decomp: finalize `cam-layout` and `cam-master`
* decomp: finalize `cam-states`
* cleanup: bi-annual formatting of the casting files
* formatting
* decomp: start working on beach-obs
* blocked: `beach-obs` mostly finished, but handle cast issues and unknown `prebind` func signature
* blocked: `villagep-obs` done, `s6` not being referred to as `self`
* decomp: finish `citadel-obs`
* decomp: finish `darkcave-obs`
* blocked: need to allow `hud-h` to decompile properly (#f as static pointer)
* decomp: finish `jungle-obs`
* decomp: finish `village-obs`
* blocked: `misty-obs` handle cast issues
* decomp: finish `village2-obs`
* decomp: 1 function left in `swamp-obs`, particle related -- maybe we know now?
* decomp: finish `swamp-obs`
* blocked: `maincave-obs` handle casts
* decomp: finish `sunken-obs`
* blocked: `rolling-obs` handle casts and hud-parts
* decomp: finish `firecanyon-obs`
* decomp: finish `ogre-obs`
* blocked: `village3-obs` gives up type analysis!
* blocked: `snow-obs` has hud-parts and handle casts code
* decomp: finish `snow-flutflut-obs`
* blocked: `lavatube-obs` has s6-1 issue
* blocked: `title-obs` handle cast issues
* fixed post merge problems
* decomp: finish `jungleb-obs`
* blocked: `training-obs` has `s6-1` issue
* fix type consistency
* scripts: Extend update script to handle the game-text-id enum as well
* git: Update git attributes to effectively halve PR burden
* fixed `sound-play-by-name` signature
* fix particle definitions in firecanyon-obs
* fix func signature in racer-states
* update ref tests
* tests: update current ref tests
* tests: add `joint` to ref-tests
* tests: add `process-drawable` to ref-tests
* updated gsrc
* add back manual fix
* address most feedback, update source files
* get rid of forward declarations in `darkcave-obs`
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
* blocked: `powerups` finicky handle casts
* stuck: decompiled `ocean-texture` but `ocean-verts` is a mystery to me
* decomp: finish `merc-death`
* fix some issues, add a way to remind us for files we've manually touched
* tests: add `ambient` to ref tests, uncover compiler error
* tests: add `main` and `font` to ref tests
* decomp: mostly finish `cam-master`
* decomp/scripts: lots of work in cam-states
* stash
* Merge remote-tracking branch 'water111/master' into decomp/camera-master
Updated submodule third-party/googletest
* decompiler: Add support for non power of 2 offsets for inline arr access
* decomp: mostly finish `cam-states` need to fix a macro issue
* blocked: `cam-master` decompiler crash when adding casts
* decomp: finish `cam-states-dbg`
* decomp: mostly finish `pov-camera` with the exception of joint-related code
* decomp: `cam-debug` finished decompiling, no way does this compile yet though
* decomp: considerable work done in `cam-layout`
* decomp: `cam-layout` almost done!
* decomp: `pov-camera` finished, TC tests will fail for now
* decomp: working on resolving issues
* decomp: cam-layout decompiling
* fixing more issues in cam-master...one event handler remains
* skip problematic function in `cam-master` for now
* gsrc: update res macros
* decomp: finish `cam-states`
* decomp: giving up on `cam-debug`
* tests: allow skipping state handlers in ref tests
* decomp: working through cam-layout bugs
* decomp: allow for shifting non-integers
* decomp: finalize `cam-layout` and `cam-master`
* decomp: finalize `cam-states`
* cleanup: bi-annual formatting of the casting files
* formatting
* address feedback - leave the float labels alone for now
* address feedback
* linting/formatting
* update gsrc and ref tests
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
* decomp `fisher`, `robotboss`, `light-eco`, `green-eco-lurker`, `sage-finalboss`, `robotboss-weapon`, `robotboss-misc`
* fixes
* add files
* add `:states` list to `deftype` and fix files
* test state forward decl's on a few more types
* also the refs
* add light-eco
* whatever
* time of day
* goal code seems to work
* stars at wrong spot
* stars and sun work
* debugging clouds
* fix texture correction
* sky works
* cleanup, add profiler
* clean up
* final clean up
* offline tests
* missing include
* blocked: `puffer` has state decomp issues
* blocked: decomp crash when adding process-drawble cast in `helix-water`
* blocked: `green-eco-lurker` no nice way to cast value out of EMB
* decomp: started on `citadel-sages`
* decomp: `seagull` blocked state decomp
* blocked: `sunken-pipegame` runs into handle related issues
* blocked: `snow-ram` and `snow-ram-boss` state decomp
* decomp: `rolling-race-ring` blocked, handle casts
* blocked: state decomp in `part-tester`
* decomp: started `joint-exploder`
* blocked: `anim-tester` has rare cmove-#f-zero usage
* decomp: I lied! `anim-tester` finished
* config: update banned list
* `flying-lurker`
* why'd this get left behind?
* also `ambient` and add a `banned_objects` config and fix offline tests
* clang
* fix `ambient-type-sound`
* mips 2 c basic version, not yet tested
* calling works without crashing, but the function doesn't
* it works
* add test
* cleanup and actually add the test
* dont use mips2c by default for font
* clean up formatting
* add script to make adding lambda label casts a bit easier
* decomp: almost finish `evil-bro`
blocked by handle -> process conversion it seems:
`No method or function named .asm.sllv.r0 for type (pointer process-tree)`
* so close! SLLV issue (not a simple handle scenario)
* decomp: finalize `evilbro` blocked by func return type
* decomp: finish `evilbro`
* add support for non virtual states
* typecheck go
* start on virtual states
* more support for virtual states
* offline passes
* fix tests
* use behavior shortcut instead of lambda
* final cleanup of virtual go
* unused var warnings and fix inconsistent enum decompile order on win vs linux
* fix thread safety bug with goal symbol table and vif1 interrupt handler
* fix type mistake
* started process on `camera`
* the bulk of `camera` is finished, with the exception of a few
* decomp: Confirmed function ret value
* decomp: `cam-combiner` dead code issue
* stash
* decomp: finish `cam-combiner`
* decomp: finish `cam-start`
* decomp: mostly finish `cam-update`, just needs polish now
* `cam-standard-event-handler` still causing issues
* `cam-combiner` issues with top-level `s6` usage
* decomp: update reference tests
* decomp: finalize `cam-start`
* decomp: `cam-update` cleaned up everything except the array of planes issue
* label correction
* still blocked in cam-combiner over casting issues
* decomp: resolve issues in `cam-start`
* decomp: finalize `cam-update`
* stash
* decomp: finalize `cam-combiner`
* decomp: finalize `camera`
* decomp: address feedback
* clean up
* before int to float stuff
* before trying to eliminate the separate read and write maps
* partial fix for register issues
* add missing include
* [decomp] `loader` prelim work + some cleanup
* more things
* more
* even more
* yet even more
* minor fixes
* decompiler fix + use behaviors for two funcs
* last functions
* Create loader_REF.gc
* more work
* change vag tool config format + unrelated farmer and yakow stuff
* update some things
* fix some decomp
* cleanup things i came across + make dgo compileable
* fix consistency test
* update refs
* offline test: skip buggy `external-art-buffer` method
* fix test
* decomp: mostly done `hint-control`
* decomp: Started and decent chunk of `menu` done
* temp stash
* decomp: escape from `menu` hell
* decomp: starting on `default-menu`
* decomp: As much as i can do in `default-menu` at this time
* decomp: clean up `hint-control`
* decomp: fix reference tests
* temp stash
* decomp: finalize `menu`
* decomp: add `menu` to goal_src
* decomp: finalize `hint-control`
* decomp: Resolve TypeConsistency issues
* and fix reference tests
* address feedback
* format and lint
* fix tests
* stash
* decompiler: support symbols in arrays
* decomp: a ton of lambda casts
* finish task control?
* add lambda label casts
* decomp: resolve some issues, add enums
* decomp: One last decompiler issue remains
* decomp: finalize `task-control`
* linting
* revert change to test change, didn't mean to commit
* decomp: comment out function call for now
* [decompiler] make certain floats decompile to constants
* update various types in `mood-h`, `time-of-day-h` and `art-h`
* fix `time-of-day-context`
* add some offline tests
* make `new-dynamic-structure` decompile at least
* formatting
* Update all-types.gc
* Update all-types.gc
* use `kmalloc-flags` for goal `kmalloc`
* cleanup some drawable stuff, add offline tests
* cleanup generic-h a bit
* make decompiler work on 16-bit static data
* cleanup ocean stuff
* fix `ocean-spheres`
* oops forgot this
* I forgot to actually make `ocean-tables`
* Make `:do-not-decompile` work on field lookup as well
* decompile `font-h` and update a vector type + minor fixes
* fix some types
* fix font-h
* fix font-h (again)
* update a script
* fixes
* Fix segfault
* Fix `PROT_NONE` page protection on windows
* set up types
* cleaned up type analysis and got things working through atomic ops
* expression working, need types
* improved types and names
* getting close
* finish up dma-disasm
* fix
