Commit graph

177 commits

Author SHA1 Message Date
Tyler Wilding a83bd49426
d/jak2: finish onin-game and resolve type-hinting (array object) elements (#2188)
Fixes https://github.com/open-goal/jak-project/issues/2047
2023-02-04 20:36:30 -05:00
water111 a0d2bce27b
Minor bug fixes (#2128)
- make sure bsp is processed on `l` levels in extraction (caused missing
remaps)
- clean up a few prints in extraction
- handle the <15 byte differences in art group files automatically (no
more errors about file naming)
- fix potential exception thrown by merc2 in a few ways: fixed bad data
in FR3's, check texture index just in case, and handle exceptions a
little bit better (still a crash, but at least you get a print)
- fix mips2 ocean stuff causing ocean far crashes
2023-01-14 16:26:17 -05:00
Tyler Wilding 6181c6c997
decomp: output mips2c method/function declarations to the _disasm.gc file (#2054)
When working with mips2c recently, I noticed adding the
`defmethod-mips2c` or `def-mips2c` code was a manual step. This is a bit
tedious to have to go and do yourself, but more importantly you have to
manually go and find the right spot in the source file else you might be
declaring it too early or too late.

This will automatically output the declaration for methods, and a
half-finished comment for the functions. I wasn't able to fully output
the function one because it seems the signature info from `all-types`
doesn't make it all the way through -- but maybe I'm wrong or this is an
easy fix?
2022-12-30 12:03:06 -05:00
Tyler Wilding 9c631e11fe
offline-test: Partition by DGO and colorize/condense output (#2045)
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular

The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it


https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
2022-12-22 13:41:33 -05:00
Tyler Wilding 190fe6968f
d/jak2: finish guard-projectile | metalhead-projectile | guard-conversation and start crimson-guard-level (#2006)
Related issue - https://github.com/open-goal/jak-project/issues/2005
2022-11-06 14:30:43 -05:00
ManDude b0e8cba6f1
[decomp2] game-save (#1988)
memory card code was not done!
2022-10-29 16:08:04 -04:00
water111 ddd60fca48
[decompiler] handle pointer to symbol value, clean up prints on offline test (#1978)
- fix issue described in
https://github.com/open-goal/jak-project/issues/1939
- fix `text`, which was manually patched with the wrong offset (was
reading the symbol value off by one byte)
- clean up some random useless prints
- make the offline tests keep trying if there's a comparison error,
clean up the output a bit so the diffs are all at the end.
2022-10-16 18:19:59 -04:00
Tyler Wilding d402ad8918
tests: parallelize offline-test execution (#1974)
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.

These changes allow us to throw hardware at the problem

Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
2022-10-16 17:20:44 -04:00
water111 e443676889
[decomp] sky-tng (#1972)
https://www.youtube.com/watch?v=fkAMvEYXOGc
2022-10-15 18:21:17 -04:00
water111 3d4dfb2077
[decomp] Decompile some time-of-day stuff, support new style Jak 2 time of day (#1943)
- Add "tfrag-water" tfrag tree support (may just be the same as Jak 1's
'dirt' for the settings)
- Add "tfrag-trans" tfrag tree support, reusing "trans-tfrag" from jak
1.
- Add a hack to `LinkedObjectFileCreation` to handle `oracle`, which is
accidentally multiply defined as a type leftover from jak 1 (an entity
in village1), and level info for jak 2.
- Add `VI1.DGO`
- add `time-of-day.gc`, and a few other stub functions so it works
- Set up some time of day stuff in GOAL for jak 2/PC renderers
- Clean up time of day in c++ renderers, support the more complicated
weight system used by jak 2 (backward compatible with jak 1, thankfully)

The mood functions now run, so this could cause problems if they rely on
stuff we don't have yet. But it seems fine for ctysluma and prison for
now.


![image](https://user-images.githubusercontent.com/48171810/194719441-d185f59c-19dc-4cd3-a5c4-00b0cfe1d6c3.png)


![image](https://user-images.githubusercontent.com/48171810/194719449-6e051bf3-0750-42e5-a654-901313dbe479.png)


![image](https://user-images.githubusercontent.com/48171810/194719455-3ca6793e-873a-449a-8e85-9c20ffeb4da3.png)


![image](https://user-images.githubusercontent.com/48171810/194719461-8f27af17-4434-4492-96cd-8c5eec6eafdf.png)


![image](https://user-images.githubusercontent.com/48171810/194719468-720715b9-985a-4acf-928c-eab948cfcb03.png)


![image](https://user-images.githubusercontent.com/48171810/194719486-bfb91e83-f6ca-4585-80ad-3b2c0cbbd5af.png)


![image](https://user-images.githubusercontent.com/48171810/194719492-df065d2f-cb5a-47e3-a248-f5317c42082f.png)


![image](https://user-images.githubusercontent.com/48171810/194719507-91e1f477-ecfe-4d6c-b744-5f24646255ca.png)
2022-10-08 13:33:03 -04:00
Tyler Wilding 4d751af38e
logs: replace every fmt::print with a lg call instead (#1368)
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.

I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.

We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.

I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.

Closes #1358
2022-10-01 11:58:36 -04:00
ManDude 9351bf782e
[decomp2] game-info, game-task and task-control (#1884)
And everything else needed for them!

A couple functions are bad currently.

- fixes #1929 - untested on linux
- fixes #1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
2022-09-27 19:44:20 -04:00
Tyler Wilding e64414d9d0
d/jak2: finish drawable-group | drawable-inline-array | drawable-tree and path (#1899) 2022-09-24 12:27:02 -04:00
ManDude cfb6abb24d
[decompiler] rewrite set lets as just sets (#1858)
Rewrites specific kinds of lets where the return value of the `set!`
itself is meant to be used to just be `set!`s. Implemented at the let
rewrite level. Seems to work for Jak 2 so far.

Fixes #1854 .
2022-09-07 18:28:01 -04:00
Tyler Wilding 44d59e6b33
d/jak2: finish mood-funcs | mood-funcs2 | almost all of mood and airlock (#1842) 2022-09-06 20:35:00 -04:00
water111 b5d21be9c5
W/misc fixes (#1838)
* temp

* temp

* before cleaning up

* cleanup merge

* fix warnings

* merge fix

* clang format
2022-09-05 20:29:12 -04:00
Tyler Wilding 71871595da
lsp: re-analyze IR2 files when they change (#1841)
* lsp: re-analyze IR2 files when they change

* tests: update ref tests
2022-09-05 17:45:41 -04:00
Tyler Wilding 01abde35d8
d/jak2: First few files, mood-tables and vol-h (#1796)
* decomp: format jak1 cast files

* decomp: finish `mood-tables` and `mood-tables2`

* jak2: stop disasm'ing

* jak2: format jak2 cast files, start working on the rest of the `mood` files

* scripts: fix running pcsx2 watcher

* d/jak2: finish `vol-h`

* d/jak2: address feedback
2022-08-24 19:34:09 -04:00
water111 06ef52cd25
[decompiler] support for jak 2 (#1781)
* [decompiler] suppport jak 2

* cleanpu

* remove brief from gtest options

* fix test
2022-08-22 18:53:51 -04:00
water111 666a19dd96
[decompiler] handle types without inspects better in all-types (#1735)
* [decompiler] handle types without inspects better in all-types

* move a few types I missed
2022-08-06 12:17:24 -04:00
Tyler Wilding b3e77c673f
decomp/lsp: Differentiate warnings from likely/definite errors (#1725)
* decomp: differentiate potential false positive warnings from likely/certain failures

* lsp: handle IR2 errors

* decomp: downgrade an expr building warning as often expressions build fine

* tests: update reference tests since comments aren't ignored

* decomp: simplify warnings interface

* tests: update ref tests
2022-08-06 11:52:36 -04:00
Tyler Wilding a66ec7c601
all-types: Improve Jak 2's all-types (#1728)
* all-types: improve all-types generation

* all-types: re-generate all-types

* tests: remove the test reporting feature

the format indeed doesn't work, and all current actions require too many permissions for forked PRs.

I'll make my own eventually that works properly (use the new markdown feature)

* all-types: put the states in the method table instead

* all-types: replace all `*time*...uint64` fields with `time-frame` type

* all-types: address feedback
2022-08-05 17:39:32 -04:00
Tyler Wilding 01c70368e3
LSP: initial LSP implementation for IR files to assist with decompilation (#1647)
* lsp: json-rpc example is working, a decent place to start...

* lsp: vendor library

* lsp: cleanup and time to get started

* lsp: commit what i got so far

* lsp: example `initialize` payload

* lsp: switch to `stdio`

* stash

* modularize the lsp implementation

* lsp: implement first actual LSP feature - function names in outline

* lsp: produce document diagnostics

* lsp: remove unused third-party lib

* lsp: support hovering MIPS instructions in IR files

* lsp: basic go-to all-types definition

* stash

* lsp: cleanup code, just need to add it to the release artifacts

* fix some project configuration

* fix linux build

* lsp: add lsp to PR artifacts and release assets

* lsp: address feedback
2022-07-18 18:26:57 -04:00
water111 dc652d10c5
[jak2] decomp gkernel, setup offline tests (#1638)
* add comments

* oops

* format'

* spelling is hard
2022-07-12 18:50:18 -04:00
water111 88166300a9
[decompiler] fix memory leak on empty pairs (#1635) 2022-07-10 19:39:38 -04:00
water111 5e23057ed1
[goalc] compile/run code for jak2 ckernel, set up dummy KERNEL.CGO (#1625)
[goalc] start can compile and run code for jak2 ckernel, set up dummy KERNEL.CGO
2022-07-08 19:23:49 -04:00
Tyler Wilding 6446389263
extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
water111 dd41640448
[decompiler] missing features in inspect method detection (#1613)
missing features in inspect method detection
2022-07-05 15:30:42 -04:00
Tyler Wilding 8fefd298fd
build: get rid of clang-cl in favor of actual clang among other things (#1589)
* git: ignore vs build dir

* cmake: ditch `clang-cl` on windows in favor of actual `clang`

* build: suppress a significant number of warnings

* build: adjust workflows and vendor nasm

* docs: update docs to remove `clang-cl` mentions

* build: fix some copy-paste mistakes in the linux build

* build: remove C++20 compat warnings as that is useful if we want to upgrade
2022-07-03 17:35:39 -04:00
water111 1e33dcda4b
Remove assets folder, use more std::filesystem (#1575)
* Remove assets folder, use more std::filesystem

* windows fix

* another one for windows

* another one

* better system for different folders

* rm debugging stuff

* let extractor override everything

* dont revert jak1 change
2022-06-29 23:32:46 -04:00
Tyler Wilding 36dc015d62
jak2: Create goal_src skeleton (#1576) 2022-06-29 22:20:09 -04:00
water111 91fa0122d8
[decompiler] Jak 2 modifications, new all-types code (#1553)
* temp

* look at old game types

* clean up
2022-06-25 21:26:15 -04:00
Tyler Wilding 2d595c1ac0
lint: add include sorting config to clang-format (#1517) 2022-06-22 23:37:46 -04:00
ManDude 7d5045ab3f
PAL & NTSC-J support + updates (#1490)
* PAL dumps files

* alrighty then

* make PAL warning slightly more specific

* PAL patches for `title-obs`

* Update all-types.gc

* PAL patch `beach-obs`

* `process-taskable` PAL patch

* `ambient` PAL patch

* `yakow` PAL patch

* `village-obs` PAL patch

* `sparticle-launcher` patch

* `swamp-obs` PAL patch

* `sequence-a-village1` PAL patch

* typo

* errors

* `powerups` PAL patch

* `ogreboss` PAL patch

* jak 1 v2 encoding

* `load-boundary` PAL patch

* `flying-lurker` PAL patch

* `mayor` PAL patch

* update game encoding to PAL (v2) encoding

* `cam-debug` and `cam-update` PAL patch

* `fisher` PAL patch

* `target` PAL patch

* `target2` PAL patch and fix text compiling

* `target-death` PAL patch

* `target-racer-h` PAL patch

* `logic-target` PAL patch

* `main` PAL patch

* `snow-flutflut-obs` PAL patch

* `rolling-obs` PAL patch

* `gsound` PAL patch

* update refs

* `progress` and `progress-draw` PAL patches

* clang

* wrong.

* complain

* clang

* fix test

* fix blurry jp text

* fix weird interrupt lag from setting window size

* patch more text lines, special handling for credits

* Update FontUtils.cpp

* Add xdelta3 and file patching interface

* add window lock toggle and update settings ver

* better particle hacks

* add PAL support to extractor

* Fix credits

* also NTSC-J support

* make xdelta3 a separate library

* address feedback

Co-authored-by: water <awaterford111445@gmail.com>
2022-06-22 05:16:34 +01:00
ManDude 7ce58f709f
process-spawn + pretty printer improvements (#1428)
* some jp support to fix some errors in the original game

* music fade toggle

* recognize `process-new` macros!!

* strip casts in this macro

* rename macro

* fix cast typecheck

* update source 1

* detect kernel stack case

* less boilerplate

* `manipy-spawn` special case

* pretty printer improvements

* revert dumb thing from earlier

* use shell detection on `send-event`

* fix some events

* remove unused argument

* detect `static-attack-info` and add `CondNoElse` to shell detect

* better `attack-info` detect

* support `process-spawn` in multi-lets

* detect `rand-float-gen` pt 1

* detect as return value

* detect in `countdown` and  `dotimes`

* oops this wasnt working

* fancier `send-event`s

* clang

* update source!!

* fix tests

* fine jeez

* uh okay

* fix some accidental regressions

* fix more regressions

* regression fixes

* fix big bug...

* extra safety!
2022-06-10 02:18:08 +01:00
water111 ecb2781a89
[decompiler] cleanup to get atomic ops working on jak2 inputs (#1426)
* [decompiler] cleanup to get atomic ops working on jak2 inputs

* clang format
2022-06-08 18:34:52 -04:00
water111 aff2f2e10c
[decompiler] setup before making IR2 type inspector (#1423) 2022-06-06 17:58:49 -04:00
water111 ec9c7a55ce
hook up merc2 to fog intensity, flip some min/max fogs (#1403)
* hook up merc2 to fog intensity, flip some min/max fogs

* zero initialize let stats
2022-06-01 19:07:28 -04:00
water111 148f52876a
[decomp] add import statements for art groups (#1372)
* new code

* update goal_src and remove all_imports

* update reference

* add decompiler list of imports

* deduplicate and move below decomp begins
2022-05-23 18:53:02 -04:00
ManDude 0212aa10c9
[decomp] better handling of animation code and art files (#1352)
* update refs

* [decompiler] read and process art groups

* finish decompiler art group selection & detect in `ja-group?`

* make art stuff work on offline tests!

* [decompiler] detect `ja-group!` (primitive)

* corrections.

* more

* use new feature on skel groups!

* find `loop!` as well

* fully fledged `ja` macro & decomp + `loop` detect

* fancy fixed point printing!

* update source

* `:num! max` (i knew i should've done this)

* Update jak1_ntsc_black_label.jsonc

* hi imports

* make compiling the game work

* fix `defskelgroup`

* clang

* update refs

* fix chan

* fix seek and finalboss

* fix tests

* delete unused function

* track let rewrite stats

* reorder `rewrite_let`

* Update .gitattributes

* fix bug with `:num! max`

* Update robotboss-part.gc

* Update goal-lib.gc

* document `ja`

* get rid of pc fixes thing

* use std::abs
2022-05-20 02:30:14 +01:00
ManDude 0cdeed9be7
minor windows fixes (#1311)
* [pp2] put `define` dest on a single line

* update source!

* Update type_analysis.cpp

* update old credits & racer code

* change clang-cl args (REALLY force avx)

* Update credits_REF.gc

* comment small unused code

* add timer to decompiler

* fix unnecessary copy-constructors (no speed increase)

* fixes

* Update expression_build.cpp

* wtf is this thing anyway

* im bored.

* clang

* fix!

* Revert "fix!"

This reverts commit 5b1ce6c718.

* Revert "clang"

This reverts commit 5e67d9ccd1.

* Revert "im bored."

This reverts commit 070e957ce8.

* Revert "Update expression_build.cpp"

This reverts commit b94d092fc5.

* Revert "fixes"

This reverts commit f3d871f60a.

* Revert "fix unnecessary copy-constructors (no speed increase)"

This reverts commit 9100725802.

* Keep the random inoffensive changes

* Revert "Update type_analysis.cpp"

This reverts commit d2456a5c75.

* Update type_analysis.cpp
2022-04-17 21:11:09 -04:00
water111 97dc0e14df
[decompiler] performance improvements in extraction (#1309)
* small speedups to extractor

* faster extraction
2022-04-15 20:40:10 -04:00
ManDude 65de778b5c
some fixes (#1308)
* Only append compiler flags in cmake

* add `RelWithDebInfo-clang` windows build

* bump serializer initial buffer size to 32mb

* extra flags for msvc

* proper anaphoric goal macros

* specify windows sdk version?

* fix "Object files are not named properly" fake error

* Update goal-lib.gc
2022-04-15 18:32:37 -04:00
Tyler Wilding c4a92571b2
Improve ASSERT macro, fix linux file paths in Taskfile and hopefully fix the windows release (#1295)
* ci: fix windows releases (hopefully)

* scripts: fix Taskfile file references for linux

* asserts: add `ASSERT_MSG` macro and ensure `stdout` is flushed before `abort`ing

* asserts: refactor all `assert(false);` with a preceeding message instances

* lint: format

* temp...

* fix compiler errors

* assert: allow for string literals in `ASSERT_MSG`

* lint: formatting

* revert temp change for testing
2022-04-12 18:48:27 -04:00
water111 f341be65e9
remove old ir1 code (#1287) 2022-04-09 11:46:56 -04:00
Tyler Wilding 8574e10653
Mark OpenGOAL regions in IR2 output and cleanup some now irrelevant settings (#1237)
* remove redundant vscode settings and duplicate goalc atoms

* decompiler: add some region markers for OpenGOAL blocks
2022-03-17 21:39:11 -04:00
Tyler Wilding b21f0d3397
Add PC Port settings to legit start menu (#1217)
* decompiler: support adding new strings to the game_text.txt file

* gsrc: expand the pckernel type and functions to work better with the menu

* gsrc: add new text-ids

* gsrc: add new macros to help with menu code

* gsrc: make a new type to generalize their list menu options

* gsrc: add new menu options and enums

* gsrc: cleanup and refactor the list menu option drawing code

this allows us to easily add a new list menu option...well as easy as the rest is atleast (setting up static lists properly, etc)

* gsrc: add and cleanup handling of new menu options

* scripts: add checks with nice error messages for user facing taskfile recipes

* lint: formatting

* address simple feedback

* gsrc: move modified files to `pc/` folder

* gsrc: revert changes to originally decompiled files

* gsrc: move modified and new files to `goal_src/pc` folder

* gsrc: update paths in `all_files.gc`
2022-03-10 19:25:01 -05:00
water111 78cde74d5a
update readme and fix always playing str (#1139)
* update readme deps

* replace assert

* bump timeout

* fix memory corruption in kernel

* use unknown if level name is invalid
2022-02-08 19:02:47 -05:00
water111 57649d243f
[bones] Part 1: Joint (#1110)
* joints

* bones

* align spr to 16k

* fix bug

* lack of sound hack

* clean up
2022-01-28 21:32:03 -05:00
ManDude 27bb6c6384
[decompiler] decompile particle defs using specialized macros (#1077)
* [decompiler] `defpart` and `defpartgroup`

* support sounds in part defs natively

* make `meters_to_string`

* update refs

* clang

* make macros in pair only work inside `(unquote ...`

* update source

* update finish

* fix
2022-01-15 20:01:38 -05:00