Commit graph

112 commits

Author SHA1 Message Date
water111 ddd60fca48
[decompiler] handle pointer to symbol value, clean up prints on offline test (#1978)
- fix issue described in
https://github.com/open-goal/jak-project/issues/1939
- fix `text`, which was manually patched with the wrong offset (was
reading the symbol value off by one byte)
- clean up some random useless prints
- make the offline tests keep trying if there's a comparison error,
clean up the output a bit so the diffs are all at the end.
2022-10-16 18:19:59 -04:00
Tyler Wilding d402ad8918
tests: parallelize offline-test execution (#1974)
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.

These changes allow us to throw hardware at the problem

Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
2022-10-16 17:20:44 -04:00
Hat Kid da5aef8d60
decomp: finish target-[util|darkjak|swim|gun] | water | water-anim | crates | dark-eco-pool, fix skelgroup detection, add failed store/load warnings and clean up jak 3 config (#1958)
Almost done:
- `target-handler` (`(none)` event handler casts and CFG error)
- `target2` (`(none)` event handler casts)
- `powerups` (`cloud-track` does some weird stuff with `handle`s)
- `gun-states` (CFG error)

Some progress in:
- `water-flow`

Additionally:

- Clean up the two year old Jak 3 config file and add a config skeleton
(disassembling seems to not have worked, but I was able to dump obj
files and the `all_scripts` file)
- Fix automatic skelgroup detection and `defskelgroup` macro for Jak 2
(closes #1950)
- When a function decompiles without any major errors, a warning is
generated with the op id for each unresolved load and store that will
likely fail to compile (closes #1933)
2022-10-14 19:35:57 -04:00
Tyler Wilding d00ebe5397
tests: allow for a single file to be tested in the offline-tests at a time (#1907)
A small quality of life increase that is more impactful since jak 2 has
double the file count.

I often use the offline-tests to find compiler errors / automatically
resolve them when i am finalizing a file. As more and more files are
completed this becomes increasingly more inefficient. When I know that
only 1 file needs to be decompiled / compared / compiled, I'd prefer to
have a feature like this.
2022-09-24 13:04:52 -04:00
water111 06ef52cd25
[decompiler] support for jak 2 (#1781)
* [decompiler] suppport jak 2

* cleanpu

* remove brief from gtest options

* fix test
2022-08-22 18:53:51 -04:00
water111 da4ec008c3
Add a gltf level exporter (#1719)
* Add a gltf level exporter

* more fixes

* disable by default

* compile fixes for windows

* improve collide packer

* bug fix

* clang format

* fix texture bug
2022-08-05 12:25:35 -04:00
Tyler Wilding 7c8ac12b55
decomp: fix calls to read_config_file (#1717) 2022-07-31 19:25:18 -04:00
Tyler Wilding d1ad6c3817
utf8: fix locale mis-handling on linux (#1698)
* utf8: fix locale mis-handling on linux

* lint: formatting
2022-07-23 10:30:23 -04:00
water111 dc652d10c5
[jak2] decomp gkernel, setup offline tests (#1638)
* add comments

* oops

* format'

* spelling is hard
2022-07-12 18:50:18 -04:00
Tyler Wilding 1b67d1dc77
Fix UTF-8 handling when running the game, env-vars, and setting the project path (#1632)
* fix utf-8 handling around env-vars

* fix file opening errors related to unicode

* add uncaught exception handler in `gk` to ensure something is logged

* gracefully fail if window icon cant be loaded and work with unicode

* linux fix and add changes to vendor file
2022-07-10 19:03:24 -04:00
water111 28a2ecdfd3
[jak2] goalc supports multiple projects (#1619)
* [jak2] goalc supports multiple projects

* disable deci2 server if not debugging
2022-07-06 21:18:08 -04:00
Tyler Wilding 8a18072d97
tests/jak2: Better prepare offline tests for multiple games (#1591)
* git: ignore vs build dir

* cmake: ditch `clang-cl` on windows in favor of actual `clang`

* build: suppress a significant number of warnings

* build: adjust workflows and vendor nasm

* docs: update docs to remove `clang-cl` mentions

* tests: move jak1 reference tests into their own folder

* tests: update offline tests to support multiple games

* tests: some additional fixes and multi-game handling

* tests: update reference tests
2022-07-06 18:10:38 -04:00
Tyler Wilding 6446389263
extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
water111 1e33dcda4b
Remove assets folder, use more std::filesystem (#1575)
* Remove assets folder, use more std::filesystem

* windows fix

* another one for windows

* another one

* better system for different folders

* rm debugging stuff

* let extractor override everything

* dont revert jak1 change
2022-06-29 23:32:46 -04:00
Tyler Wilding 36dc015d62
jak2: Create goal_src skeleton (#1576) 2022-06-29 22:20:09 -04:00
Tyler Wilding 2d595c1ac0
lint: add include sorting config to clang-format (#1517) 2022-06-22 23:37:46 -04:00
water111 ecb2781a89
[decompiler] cleanup to get atomic ops working on jak2 inputs (#1426)
* [decompiler] cleanup to get atomic ops working on jak2 inputs

* clang format
2022-06-08 18:34:52 -04:00
ManDude 0212aa10c9
[decomp] better handling of animation code and art files (#1352)
* update refs

* [decompiler] read and process art groups

* finish decompiler art group selection & detect in `ja-group?`

* make art stuff work on offline tests!

* [decompiler] detect `ja-group!` (primitive)

* corrections.

* more

* use new feature on skel groups!

* find `loop!` as well

* fully fledged `ja` macro & decomp + `loop` detect

* fancy fixed point printing!

* update source

* `:num! max` (i knew i should've done this)

* Update jak1_ntsc_black_label.jsonc

* hi imports

* make compiling the game work

* fix `defskelgroup`

* clang

* update refs

* fix chan

* fix seek and finalboss

* fix tests

* delete unused function

* track let rewrite stats

* reorder `rewrite_let`

* Update .gitattributes

* fix bug with `:num! max`

* Update robotboss-part.gc

* Update goal-lib.gc

* document `ja`

* get rid of pc fixes thing

* use std::abs
2022-05-20 02:30:14 +01:00
water111 97dc0e14df
[decompiler] performance improvements in extraction (#1309)
* small speedups to extractor

* faster extraction
2022-04-15 20:40:10 -04:00
Tyler Wilding ab063bf7b0
extractor: split up extraction process and allow overriding data dir path (#1302)
* extractor: split up extraction process and allow overriding `data` dir path

* lint: formatting

* deps: add CLI11 dependency

* extractor: refactor CLI arg handling
2022-04-15 18:01:47 -04:00
water111 f341be65e9
remove old ir1 code (#1287) 2022-04-09 11:46:56 -04:00
water111 5bd0b735a5
Add extractor tool (#1276)
* first attempt

* fix

* zip to tar

* windows

* try again, std::filesystem sucks

* std::filesystem is still garbage

* std::filesystem is terrible

* std::filesystem continues to waste my time

* again

* neadsflaldksal;df
2022-04-03 19:17:03 -04:00
Tyler Wilding ffb04ddd10
Documentation cleanup and some feature improvements (#1155)
* ci: switch to codacy for coverage

* docs: update badges

* decomp: allow overriding config flags via CLI

* cleanup: top level file cleanup

* docs: big README overhaul

Attempt to close #1128 and #1086

* decomp: attempt to detect if `iso_data` is missing or wrongly extracted

* game: switch to `fpng` for screenshots, allow for compression

closes #1035

* game: switch vsync control to a checkbox

* lint: format cpp files

* lint: format json files

* docs/scripts: organize taskfile
2022-02-12 17:48:50 -05:00
water111 78cde74d5a
update readme and fix always playing str (#1139)
* update readme deps

* replace assert

* bump timeout

* fix memory corruption in kernel

* use unknown if level name is invalid
2022-02-08 19:02:47 -05:00
water111 bcbd1159dd
Continue cleanup of source files (#1138)
* more cleanup

* tests
2022-02-06 18:53:43 -05:00
water111 54c6ddbab9
pass filename through (#1080) 2022-01-15 23:31:07 -05:00
Tyler Wilding 45318be063
decomp: finish _almost all of_ the remaining camera code (#845)
* decomp: mostly finish `cam-master`

* decomp/scripts: lots of work in cam-states

* stash

* Merge remote-tracking branch 'water111/master' into decomp/camera-master

Updated submodule third-party/googletest

* decompiler: Add support for non power of 2 offsets for inline arr access

* decomp: mostly finish `cam-states` need to fix a macro issue

* blocked: `cam-master` decompiler crash when adding casts

* decomp: finish `cam-states-dbg`

* decomp: mostly finish `pov-camera` with the exception of joint-related code

* decomp: `cam-debug` finished decompiling, no way does this compile yet though

* decomp: considerable work done in `cam-layout`

* decomp: `cam-layout` almost done!

* decomp: `pov-camera` finished, TC tests will fail for now

* decomp: working on resolving issues

* decomp: cam-layout decompiling

* fixing more issues in cam-master...one event handler remains

* skip problematic function in `cam-master` for now

* gsrc: update res macros

* decomp: finish `cam-states`

* decomp: giving up on `cam-debug`

* tests: allow skipping state handlers in ref tests

* decomp: working through cam-layout bugs

* decomp: allow for shifting non-integers

* decomp: finalize `cam-layout` and `cam-master`

* decomp: finalize `cam-states`

* cleanup: bi-annual formatting of the casting files

* formatting

* address feedback - leave the float labels alone for now

* address feedback

* linting/formatting

* update gsrc and ref tests

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
2021-10-16 21:01:23 -04:00
water111 f87646e8ce
Small state fixes (#901)
* small fixes

* catch uncaught exception
2021-10-15 21:31:22 -04:00
water111 3a1c9eaf75
[decompiler] clean up offline test program, reorganize decomp order (#895)
* make a new offline test

* finish up offline test and fix crash bugs
2021-10-12 20:33:26 -04:00
water111 b2052016e2
[decomp] sky/time of day (#883)
* time of day

* goal code seems to work

* stars at wrong spot

* stars and sun work

* debugging clouds

* fix texture correction

* sky works

* cleanup, add profiler

* clean up

* final clean up

* offline tests

* missing include
2021-10-10 20:07:03 -04:00
Tyler Wilding 2cb00f4a02
decomp: anim-tester and a bunch of progress on files that are blocked (#884)
* blocked: `puffer` has state decomp issues

* blocked: decomp crash when adding process-drawble cast in `helix-water`

* blocked: `green-eco-lurker` no nice way to cast value out of EMB

* decomp: started on `citadel-sages`

* decomp: `seagull` blocked state decomp

* blocked: `sunken-pipegame` runs into handle related issues

* blocked: `snow-ram` and `snow-ram-boss` state decomp

* decomp: `rolling-race-ring` blocked, handle casts

* blocked: state decomp in `part-tester`

* decomp: started `joint-exploder`

* blocked: `anim-tester` has rare cmove-#f-zero usage

* decomp: I lied! `anim-tester` finished

* config: update banned list
2021-10-10 19:53:19 -04:00
Tyler Wilding fbcc1542f3
decomp: decompile remaining particle files (#865)
* decomp: finish `beach-part`

* decomp: finish `citadel-part`

* decomp: finish `robotboss-part`

* decomp: finish `sage-finalboss-part`

* decomp: finish `jungle-part`

* decomp: finish `flut-part`

* decomp: finish `firecanyon-part`

* decomp: finish `village2-part2`

* decomp: finish `swamp-part`

* decomp: finish `maincave-part`

* decomp: finish `sunken-part0-5`

* decomp: finish `ogre-part`

* decomp: finish `robocave-part` and `snow-part`

* decomp: finish `collectables-part`

* decomp: finish `village1-part`

* decomp: finish `village1-part2

* decomp: finish `training-part`

* decomp: finish `misty-part`

* decomp: finish `village2-part`

* decomp: finish `village3-part`

* decomp: finish `lavatube-part`

* decomp: finish `hud-h`, `target-racer-h`, `racer-part`

* address feedback
2021-09-29 20:33:40 -04:00
water111 7ebbe812b1
Cleanup sparticle, add ref tests, name a few methods (#863)
* temp

* add reference tests
2021-09-28 21:17:38 -04:00
Tyler Wilding dd959d0d0d
decomp: Continuing full pass of gameplay code (#839)
* decomp: finish `babak` - :code is called in `(code nav-enemy-patrol babak)`

* decomp: almost finish `process-taskable`

* blocked: mistycannon / pelican

* decomp: finish `babak-with-cannon` write a script to fix gsrc

* decomp: finish `process-taskable`

* decomp: finish `flutflut` and `yakow`

* decomp: finish `fishermans-boat`

* blocked: state decomp `training-obs`

* decomp: finish `muse`

* decomp: finish `bonelurker`

* blocked: state decomp in `quicksandlurker`| `balloonlurker`

* decomp: finish `assistant-village2`

* scripts: script to help updating goal_src

* starting to update goal_src

* tests: update ref tests

* src: more src updating

* src: waiting on `process-taskable` and `muse`

* blocked: `citb-plat` state decomp

* decomp: finish `square-platform`

* blocked: `orbit-plat` due to overlays + static data

* decomp: finish `qbert-plat`

* blocked: almost finish `misty-conveyor`, sparticle-callback

* blocked: jungle-mirrors

* blocked: state decomp in `swamp-blimp`

* decomp: finish `swamp-bat`

* decomp: finish `swamp-rat`

* decomp: finish `swamp-rat-nest`

* blocked: state decomp `kermit`

* decomp: finish `cavecrystal-light`

* decomp: finish `spiderwebs`

* blocked: state decomp `dark-crystal`

* decomp: finish `baby-spider`

* decomp: finish `mother-spider-h`

* decomp: finish `mother-spider-proj`

* blocked: state decomp in `gnawer`

* blocked: state decomp in `driller-lurker`

* blocked: `sun-exit-chamber` breaks when adding handle cast

* decomp: finish `sunken-water`

* blocked: `target-tube` ShortCircuitElement::push_to_stack

* decomp: finish `sunken-fish`

* blocked: `minecart` decomp crash when adding stack cast

* decomp: finish `assistant-village3`

* decomp: finish `sage-village3`

* blocked: `cave-trap` done but ran into `go` issue

* blocked: `spider-egg` state decomp

* decomp: finish `target-snowball`

* blocked/stuck: `target-ice` decomp issue around cpad

* pausing: ice-cube has some weird collide-shape-prim handling

* blocked: `snow-ball` state decomp

* blocked: `snow-bumper` state decomp

* decomp: finish `snow-ram-h`

* decomp: finish `yeti`

* decomp: finish `assistant-lavatube`

* re-enable the float cast log

* decomp: updating to new sparticle definitions

* decomp: address feedback up to `swamp-rat-nest`

* address remaining feedback

* all-types: move the `pointer` def

* add back temporary `hud-hidden?`
2021-09-28 20:42:00 -04:00
water111 188373a3f6
decompile some drawable stuff and fix a few small bugs (#859) 2021-09-28 19:24:09 -04:00
ManDude d8e659df66
[decomp] flying-lurker (#846)
* `flying-lurker`

* why'd this get left behind?

* also `ambient` and add a `banned_objects` config and fix offline tests

* clang

* fix `ambient-type-sound`
2021-09-21 18:40:38 -04:00
water111 5683f04046
[decomp] handle handle->process inside an and (#851)
* fix the loader example

* improve compiler error message

* fix missing cast issue
2021-09-21 00:12:37 -04:00
water111 f9d8fcd6e4
[decomp] add mips2c converter (#842)
* mips 2 c basic version, not yet tested

* calling works without crashing, but the function doesn't

* it works

* add test

* cleanup and actually add the test

* dont use mips2c by default for font

* clean up formatting
2021-09-11 20:52:35 -04:00
Tyler Wilding 4ff2130d55
decomp: decompile almost all the platform related files (#812)
* deocmp: `plat` finished, `baseplat` started

* decomp: finish `baseplat`

* decomp: finish `plat-button`

* decomp: finish `plat-eco`

* decomp: finish `citb-drop-plat-CIT`

* decomp: finish `cit-drop-plat-L1`

* decomp: finish `plat-flip`

* decomp: added a bunch of label defs

* decomp: `plat-button` ref test added

* stash

* decomp: finish `baseplat`

* decomp: finish `plat`

* decomp: finish `plat-button`

* tests: fix offline-test to handle multi-DGO files better

* decomp: finish `citb-drop-plat-CIT`

* decomp: finish `plat-flip`

* decomp: finish `wedge-plats`

* tools: fix memory analyzer, skip invalid type

* decomp: finish `wall-plat`

* fix ordering

* fix some casting issues

* cleanup after conflict resolution

* address reviews
2021-09-06 21:10:19 -04:00
Tyler Wilding d1f16b34ff
decomp: another batch of easy files (#823)
* decomp: finish `sharkey`

* decomp: finish `lurkercrab`

* decomp: finish `lurkerpuppy`

* decomp: finish `assistant-citadel`

* decomp: finish `static-screen`

* decomp: finish `robotboss-h`

* decomp: finish `final-door`

* decomp: finish `aphid`

* addressed review
2021-09-06 15:34:31 -04:00
Tyler Wilding 42f7a675b2
decomp: a few files that were done except for sp-launch-group (#802)
* decomp: dark-eco-pool blocked by sparticle-launch-group label

* decomp: finalize `dark-eco-pool`

* decomp: mostly finish `beach-rocks` waiting for sparticle label supp

* decomp: almost done `beach-rocks` but potential new handle case

* decomp: finish `beach-rocks`

* decomp: finish `assistant`

* decomp: finish `miners`
2021-09-01 11:20:05 -04:00
water111 d9f9e076af
[decompiler] automatically label things when possible (#784)
* improve label system

* clean up menu

* debug menu working, still need to fix tests

* fix tests and clean up
2021-08-29 11:13:06 -04:00
Tyler Wilding 4c61343cb7
decomp: evilbro (#774)
* add script to make adding lambda label casts a bit easier

* decomp: almost finish `evil-bro`

blocked by handle -> process conversion it seems:
`No method or function named .asm.sllv.r0 for type (pointer process-tree)`

* so close! SLLV issue (not a simple handle scenario)

* decomp: finalize `evilbro` blocked by func return type

* decomp: finish `evilbro`
2021-08-22 21:03:47 -04:00
water111 403bb5f4de
[decomp] game-info (#779)
* support more process stuff

* more of game info

* add ref file

* progress on save
2021-08-22 20:12:47 -04:00
water111 97ab6a4e12
add support for non virtual states (#764)
* add support for non virtual states

* typecheck go

* start on virtual states

* more support for virtual states

* offline passes

* fix tests

* use behavior shortcut instead of lambda

* final cleanup of virtual go

* unused var warnings and fix inconsistent enum decompile order on win vs linux

* fix thread safety bug with goal symbol table and vif1 interrupt handler

* fix type mistake
2021-08-17 20:54:03 -04:00
water 5aa15afe29 decompile sprite-distort without vu1 stuff 2021-08-13 18:50:29 -04:00
Tyler Wilding a9816f9d28
decomp: finish misty-teetertotter 2021-08-08 17:00:51 -04:00
Tyler Wilding d400ec3b2d
decomp: mood (#672)
* decomp: Almost finished `mood`, but confused over the `mood-context` type!

* only function that remains involves vftoi / ppach instructions

* stash

* decomp: fix inline array access issue

* decomp: finalize `mood`

* linting
2021-08-01 16:11:22 -04:00
Tyler Wilding d37e6c548e
decomp: camera | cam-combiner | cam-start | cam-update (#621)
* started process on `camera`

* the bulk of `camera` is finished, with the exception of a few

* decomp: Confirmed function ret value

* decomp: `cam-combiner` dead code issue

* stash

* decomp: finish `cam-combiner`

* decomp: finish `cam-start`

* decomp: mostly finish `cam-update`, just needs polish now

* `cam-standard-event-handler` still causing issues

* `cam-combiner` issues with top-level `s6` usage

* decomp: update reference tests

* decomp: finalize `cam-start`

* decomp: `cam-update` cleaned up everything except the array of planes issue

* label correction

* still blocked in cam-combiner over casting issues

* decomp:  resolve issues in `cam-start`

* decomp: finalize `cam-update`

* stash

* decomp: finalize `cam-combiner`

* decomp: finalize `camera`

* decomp: address feedback
2021-07-30 22:18:35 -04:00
water111 bd9f550d82
add ref test for sprite and support nested bitfields in static data (#723) 2021-07-26 20:02:16 -04:00