Commit graph

385 commits

Author SHA1 Message Date
Tyler Wilding abf61a94fb
docs: add support for :override-doc in method declarations as well as documenting state handlers (#2139)
Adding support for better child-type method docstrings. This is a
problem unique to methods.

Usually, a child-type will have the same signature and a common name
will apply, but the implementation is different. This means, you
probably want a different docstring to describe what is happening.

Currently this is possible to do via `:replace`. The problem with
replace is two fold:
- a replaced method ends up in the generated `deftype`...because you
usually change the signature!
- we don't put docstrings in the `deftype` in normal GOAL, this is just
something we do for the `all-types` file (they go in the `defmethod`
instead)
- more importantly, this means anytime you now want to change the
parent's name/args/return type -- you have to apply that change
everywhere.

So this is a better design you can now just declare the method like so:
```clj
(:override-doc "my new docstring" <method_id>)
```

And internally a pseudo-replaced method will be added, but it will
inherit everything from the parent (except the docstring of course)

Unrelated - I also made all the keyword args for declaring methods not
depend on ordering

This also adds support for documenting virtual and non-virtual state
handlers. For example:

```clj
  (:states
    (part-tester-idle (:event "test") symbol))
```

or

```clj
(idle () _type_ :state (:event "test") 20)
```

I will probably add the ability to give some sort of over-view docstring
at a later date.

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-21 20:45:45 -05:00
water111 7d7625f4f8
[jak2] workaround for cutscene playing (#2131) 2023-01-16 13:37:29 -05:00
Tyler Wilding dd0a8a17b2
docs - first chunk of work documenting the files I glossed over (#2130)
- started documenting the files I glossed over, some are totally done,
others are just partially done
- I changed the decompiler to automatically initialize the
art-group-info from the json file. This makes updating gsrc, even a
single file at a time, have consistent naming
- Though I disabled this functionality for jak 1, as I have no idea if
using the ntsc art groups will cause a regression for different versions
- fix indentation for docstrings -- it still doesn't look great, but
this is now a formatting concern, rather than the docstring having a
bunch of happen-stance leading whitespace.
2023-01-15 11:33:39 -05:00
water111 a0d2bce27b
Minor bug fixes (#2128)
- make sure bsp is processed on `l` levels in extraction (caused missing
remaps)
- clean up a few prints in extraction
- handle the <15 byte differences in art group files automatically (no
more errors about file naming)
- fix potential exception thrown by merc2 in a few ways: fixed bad data
in FR3's, check texture index just in case, and handle exceptions a
little bit better (still a crash, but at least you get a print)
- fix mips2 ocean stuff causing ocean far crashes
2023-01-14 16:26:17 -05:00
Tyler Wilding 8b12553d15
d/jak2: first pass at decompiling the core vehicle/traffic code (#2058)
Effects the following files:
- [x] vehicle-rider
- [x] vehicle-control
- [x] vehicle-effects
- [x] vehicle
~~- [ ] vehicle-util~~
- [x] vehicle-physics
- [x] vehicle-states
~~- [ ] vehicle-guard~~
~~- [ ] traffic-engine~~
~~- [ ] traffic-manager~~

With the exception of traffic-engine, most of these files are either
done or have 1-3 stubborn functions remaining. Draft while I try to
resolve as many as possible / cleanup names and such.

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-14 12:01:33 -05:00
Tyler Wilding 00ac12094e
goalc/repl: cleanup of goalc/REPL code and some QoL improvements (#2104)
- lets you split up your `startup.gc` file into two sections
  - one that runs on initial startup / reloads
  - the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-07 11:24:02 -05:00
Tyler Wilding 2f4146d469
tests: make the offline tests aware of the current terminals row count (#2105)
This fixes the hideous output when your terminal would be too small to
hold all the threads.
2023-01-07 10:35:12 -05:00
Tyler Wilding 22ec32ce94
tools: add a flag to just dump all types from all-types via type searcher (#2098) 2023-01-04 20:27:51 -05:00
water111 0e43c96e52
[decomp] hud, hud-classes, add some new basic alignment options, fix type bug (#2084) 2023-01-02 10:05:22 -05:00
Tyler Wilding 9f53edae7a
game/debugging: Add a new imgui menu to filter debug text and adjust imgui config settings (#2085)
This adds a new ImGUI menu to help filter out the clutter on screen.


https://user-images.githubusercontent.com/13153231/210192912-b1c28319-bacb-449c-ad7f-e7308fb75f50.mp4

This also:
- moves the imgui display bool into a game specific config file (you can
hide it in jak1, and not in jak2)
- the config file also persists the settings from this menu (except the
filters for now, future TODO)
- there is a new `ignore_imgui_hide_keybind` in this file to ignore
hiding it when you press Alt
2023-01-02 09:45:38 -05:00
water111 308038a20f
[decomp] collectables, fix float bug in a few files (#2082)
Fixes https://github.com/open-goal/jak-project/issues/1931
Also decompile collectables.gc
2023-01-01 18:05:31 -05:00
Tyler Wilding 73d38bd9ff
game/debugger: handle uncaught loader thread exceptions and better handle that situation on the windows debugger (#2075) 2023-01-01 12:31:12 -05:00
Tyler Wilding 6aa15c897f
decomp: don't mutate allowed/banned object lists (#2073)
oops, leads to some very confusing behaviour
2022-12-30 23:57:01 -05:00
Tyler Wilding 997c1ab60a
decomp: add explicit failures for common config mistakes related to allowed_objs (#2062)
A common thing that can be forgotten about / confusing to new people is
that:
1. if you add an object to the `allowed_objs` list but it's also in the
`banned_objs` list -- the ban still takes precedence with no failure
2. if you add an object to the `allowed_objs` list but have not allowed
the DGO in `inputs.jsonc` it will also silently log a failure and
continue.

This PR turns both situations into an explicit error with advice/a
reminder on what to do to fix it.
2022-12-30 14:01:55 -05:00
Tyler Wilding 2eca614870
d/jak2: finish remaining *-h files that aren't in progress or spoken for (#2059)
This finishes off all the remaining "header" files that someone isn't
already working on.
2022-12-30 12:09:04 -05:00
Tyler Wilding 216e73b61f
CI: Add a macOS Github runner (#2064) 2022-12-22 18:12:59 -05:00
water111 73561f10a3
support c++ tools on macos (#2063)
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
2022-12-22 17:12:05 -05:00
Tyler Wilding 9c631e11fe
offline-test: Partition by DGO and colorize/condense output (#2045)
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular

The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it


https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
2022-12-22 13:41:33 -05:00
water111 70e231fa72
[jak 2] merc (#2039) 2022-11-30 22:36:09 -05:00
Tyler Wilding ac3c4e59b0
goalc/repl: Allow hot-loading files via ml with just the object name (#2036)
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.


![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)

As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)

This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
  "numConnectToTargetAttempts": 1,
  "jak2": {
    "asmFileSearchDirs": [
      "goal_src/jak2"
    ]
  }
}
```

This also provides a way to make game-specific configurations for the
REPL fairly easily.
2022-11-29 19:22:22 -05:00
Tyler Wilding a0a85eb60a
repl: Add a few new quality of life improvements (#2030)
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here


https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.

If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
    "-workbench.action.quickOpen",
    "-workbench.action.quickOpenView"
  ]
```
2022-11-20 14:28:41 -05:00
Tyler Wilding bf83f2442d
d/jak2: cleanup more of editable and editable-player (#2029)
- Rough start of the SQLite integration to facilitate the SQL queries
- Cleanup and disable a little bit of code so the game no longer crashes
when entering the editor
- Implement some of the mouse data stuff


![image](https://user-images.githubusercontent.com/13153231/202881481-95bc0a2a-ac3d-4f65-aff1-b9f7ee5ee345.png)


https://user-images.githubusercontent.com/13153231/202881484-399747e7-dcdb-4e09-93e9-b561a45c8a18.mp4

This is a very old branch so best to get it merged now that it's at a
decent point so it can be iterated on.
2022-11-19 23:28:20 -05:00
Tyler Wilding 190fe6968f
d/jak2: finish guard-projectile | metalhead-projectile | guard-conversation and start crimson-guard-level (#2006)
Related issue - https://github.com/open-goal/jak-project/issues/2005
2022-11-06 14:30:43 -05:00
ManDude b0e8cba6f1
[decomp2] game-save (#1988)
memory card code was not done!
2022-10-29 16:08:04 -04:00
Tyler Wilding d402ad8918
tests: parallelize offline-test execution (#1974)
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.

These changes allow us to throw hardware at the problem

Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.
2022-10-16 17:20:44 -04:00
water111 46f11b1b47
add level ref test (#1973)
There's a few functions that have to be excluded, but I think it's still
useful to have this.
2022-10-15 18:36:32 -04:00
Tyler Wilding da82cb87e7
tools: add the ability to search by method id (#1964)
Useful when you are trying to find a type with atleast a method at a
given id.
2022-10-14 19:10:38 -04:00
Tyler Wilding f6bdc07990
d/jak2: finish progress menu code and initialize the camera (#1945)
This PR does a few main things:
- finish decompiling the progress related code
- implemented changes necessary to load the text files end-to-end
   - japanese/korean character encodings were not added
- finish more camera code, which is required to spawn the progress menu
/ init the default language settings needed for text
  - initialized the camera as well

Still havn't opened the menu as there are a lot of checks around
`*target*` which I havn't yet gone through and attempted to comment out.
2022-10-11 18:30:26 -04:00
ManDude af5009a29e
[decomp] make defpart and defpartgroup work in jak 2 (#1947) 2022-10-09 12:53:44 -04:00
water111 405a144815
[decomp] finish up debug.gc, get boundary rendering working (#1944)
![image](https://user-images.githubusercontent.com/48171810/194726152-74167185-0297-4982-8ed9-42936ad80fe7.png)
2022-10-08 17:02:40 -04:00
water111 3d4dfb2077
[decomp] Decompile some time-of-day stuff, support new style Jak 2 time of day (#1943)
- Add "tfrag-water" tfrag tree support (may just be the same as Jak 1's
'dirt' for the settings)
- Add "tfrag-trans" tfrag tree support, reusing "trans-tfrag" from jak
1.
- Add a hack to `LinkedObjectFileCreation` to handle `oracle`, which is
accidentally multiply defined as a type leftover from jak 1 (an entity
in village1), and level info for jak 2.
- Add `VI1.DGO`
- add `time-of-day.gc`, and a few other stub functions so it works
- Set up some time of day stuff in GOAL for jak 2/PC renderers
- Clean up time of day in c++ renderers, support the more complicated
weight system used by jak 2 (backward compatible with jak 1, thankfully)

The mood functions now run, so this could cause problems if they rely on
stuff we don't have yet. But it seems fine for ctysluma and prison for
now.


![image](https://user-images.githubusercontent.com/48171810/194719441-d185f59c-19dc-4cd3-a5c4-00b0cfe1d6c3.png)


![image](https://user-images.githubusercontent.com/48171810/194719449-6e051bf3-0750-42e5-a654-901313dbe479.png)


![image](https://user-images.githubusercontent.com/48171810/194719455-3ca6793e-873a-449a-8e85-9c20ffeb4da3.png)


![image](https://user-images.githubusercontent.com/48171810/194719461-8f27af17-4434-4492-96cd-8c5eec6eafdf.png)


![image](https://user-images.githubusercontent.com/48171810/194719468-720715b9-985a-4acf-928c-eab948cfcb03.png)


![image](https://user-images.githubusercontent.com/48171810/194719486-bfb91e83-f6ca-4585-80ad-3b2c0cbbd5af.png)


![image](https://user-images.githubusercontent.com/48171810/194719492-df065d2f-cb5a-47e3-a248-f5317c42082f.png)


![image](https://user-images.githubusercontent.com/48171810/194719507-91e1f477-ecfe-4d6c-b744-5f24646255ca.png)
2022-10-08 13:33:03 -04:00
Tyler Wilding be1e40a041
d/jak2: decompile conveyor | elevator | plat | bouncer | basebutton | base-plat | most of sampler | simple-nav-sphere | simple-focus | elec-gate and blocking-plane (#1942)
Most of these have been atleast partially documented / named / cleaned
up as well.

Co-authored-by: water <awaterford111445@gmail.com>
2022-10-08 12:42:52 -04:00
water111 6bd4649fd0
fix deadlock on shutdown (#1937) 2022-10-01 12:29:08 -04:00
Tyler Wilding 4d751af38e
logs: replace every fmt::print with a lg call instead (#1368)
Favors the `lg` namespace over `fmt` directly, as this will output the
logs to a file / has log levels.

I also made assertion errors go to a file, this unfortunately means
importing `lg` and hence `fmt` which was attempted to be avoided before.
But I'm not sure how else to do this aspect without re-inventing the
file logging.

We have a lot of commented out prints as well that we should probably
cleanup at some point / switch them to trace level and default to `info`
level.

I noticed the pattern of disabling debug logs behind some boolean,
something to consider cleaning up in the future -- if our logs were more
structured (knowing where they are coming from) then a lot this
boilerplate could be eliminated.

Closes #1358
2022-10-01 11:58:36 -04:00
ManDude 9351bf782e
[decomp2] game-info, game-task and task-control (#1884)
And everything else needed for them!

A couple functions are bad currently.

- fixes #1929 - untested on linux
- fixes #1924 - now you need to type `,` before a lambda you want to put
in a pair.
- fix debugger symbol table in jak 2
- made the decompiler output `(meters 2)` instead of `(meters 2.0)`
- fixed a bug with the bitfield enum special -1 case
- made bad game text decomp not exit the decompiler
- added `editable-player` and `script`
2022-09-27 19:44:20 -04:00
Tyler Wilding 332f0b2f2b
tools: add a tool to search for types based on size / type chain / fields (#1906)
Just a small simple tool that can search through `all-types` for a type
based on a bunch of criteria.

For example:

![image](https://user-images.githubusercontent.com/13153231/192043561-181e5c5d-d5b1-41a9-8891-5cc3ed1a0efa.png)

The results are printed to stdout, as well as output to a json file so
they can be consumed by another tool (in my plans, the VSCode extension)
2022-09-24 16:04:52 -04:00
Tyler Wilding 0f4f70e66d
logs: prefer exceptions over stdout logs around defenums (#1887)
This was causing an issue in the LSP because the `fmt::print`s clash
with the transport over stdin/out
2022-09-16 20:27:22 -04:00
Tyler Wilding 4620f96574
d/jak2: mostly finish mysql-nav-graph and fix docstring handling around with-pp/local-vars (#1869) 2022-09-13 18:15:02 -04:00
water111 4eea31c3e9
[jak 2] texture (#1866)
- Decompile and patch `texture.gc` for PC
- Improve decompiler when offset doesn't fit in immediate (for types
larger than 8k and some scratchpad accesses)
- Fix symbol->string issues in both jak 1 and 2
- Fix bug with VIF interrupt used to profile VU code (hooked up to
OpenGLRenderer BucketRenderers in PC port)
- Support `~o` in `format`.
- Uncomment stuff in `merc.gc` that now works!

![image](https://user-images.githubusercontent.com/48171810/189505469-941b4a3e-23c7-4740-aa1b-2e461ed19fa9.png)

fixes https://github.com/open-goal/jak-project/issues/1850
2022-09-11 14:17:55 -04:00
water111 f3c63f26bb
fix let* format, new on stack guessing case, type failure, handle casts (#1860)
Fixes https://github.com/open-goal/jak-project/issues/1821 by adding a
special case for `new` method calls where the argument with type
`symbol` is actually an address to uninitialized structure on the stack.

Fixes https://github.com/open-goal/jak-project/issues/1849 (or at least
the cause of the issue Vaser gave in chat, and one random one I found in
`debug-sphere`)

Fixes https://github.com/open-goal/jak-project/issues/1853

Fixes https://github.com/open-goal/jak-project/issues/1857 by moving the
cast into the cond if the body is a single form and the destination type
is a bitfield/enum which is likely to work well. Seems to work on the
examples we could find in jak 1 and jak 2.

Also fixes an issue with casts on the result of `handle->process` (a
common place to use casts)

the output of process->handle is a plain process. Most of the time, you
end up casting this to a more specific. If you add a cast on every use
of the variable, the decompiler will decide to change the type of that
variable to the more specific type, and this breaks the handle cast.

so previously it was impossible to get code like
```
    (let* ((s2-0 (the-as swingpole (handle->process (-> self control hack))))
           (gp-0 (-> s2-0 dir))
           )
```
But now it will work
2022-09-07 21:58:09 -04:00
water111 b5d21be9c5
W/misc fixes (#1838)
* temp

* temp

* before cleaning up

* cleanup merge

* fix warnings

* merge fix

* clang format
2022-09-05 20:29:12 -04:00
Tyler Wilding 0896bef2bf
jak1/speedruns: Some final touches for speedrunning in jak 1 (#1830)
* jak1: put common speedrunning code into it's own file

* jak1: enforce `60` fps while in speedrunning mode

* jak1: when speedrunning, display the version until you get the first powercell

* jak1: add an explicit option for skipping cutscenes

* jak1: extend `game-option` to allow any menu option to be disabled

* tests/jak1: allow whitelisting types to be redefined to satisfy typeconsistency checks

* jak1: add file headers

* jak1: cleanup bool checking

* test: delete the es substitle file

* test: add it back

* jak1: missed one cleanup spot related to bool comparisons
2022-09-02 18:15:42 -04:00
water111 97637d7902
fix for windows newlines (#1834) 2022-09-01 22:44:43 -04:00
water111 2bead6db97
[decompile] Joint (#1813)
* temp

* decompile joints

* fix merge
2022-08-28 16:38:58 -04:00
water111 f8007cc84b
[decomp] Jak 2 mips2c, collide-func (#1805) 2022-08-26 18:03:48 -04:00
Tyler Wilding 01abde35d8
d/jak2: First few files, mood-tables and vol-h (#1796)
* decomp: format jak1 cast files

* decomp: finish `mood-tables` and `mood-tables2`

* jak2: stop disasm'ing

* jak2: format jak2 cast files, start working on the rest of the `mood` files

* scripts: fix running pcsx2 watcher

* d/jak2: finish `vol-h`

* d/jak2: address feedback
2022-08-24 19:34:09 -04:00
sardap 5777b6e058
Added screenshot hotkey (#1776)
* Added screenshot hotkey and added a toggle to debug GUI to disable screenshot hotkey (#1765)

* Renamed gfx_dumps folder to screenshots and screenshots taken with hotkey now use game render options not debug gui screenshot render options (#1765)

* Updated get current timestamp to use strftime

* fixed clang formating errors for screenshot hotkey

Co-authored-by: sardap <sardap@users.noreply.github.com>
2022-08-23 19:13:26 -04:00
Tyler Wilding 090b2984a2
decomp: add docstring support to relevant places in all-types (#1753)
* decomp: add `docstring` support to relevant places in `all-types`

* decomp: output method docstring into the `defmethod` instead

* goalc: handle docstrings in `define[-extern]` (gracefully ignore for now)

* decomp: output docstrings for bitfield deftypes too

* goalc: fix `defenum` parsing when coming from the compiler (no symbol metadata to store to)

* lsp/tests: fix ups

* lint: formatting

* goalc: handle edge-case of defining a string constant

* cleanup leftovers, fix codacy issues, rename struct
2022-08-23 00:32:07 -04:00
water111 06ef52cd25
[decompiler] support for jak 2 (#1781)
* [decompiler] suppport jak 2

* cleanpu

* remove brief from gtest options

* fix test
2022-08-22 18:53:51 -04:00
Tyler Wilding dec7da2110
game: windowed mode related fixes (#1778)
* game: restore windowed mode settings properly

* game: use the game version for saving settings/saves

* game: prevent windowed mode from being auto-centered on initial init

* game: save and restore window coordinates

* lint: formatting
2022-08-21 18:13:27 -04:00