Trying to make up for some of the startup speed lost in the SDL
transition. This saves about 1s from start (from ~3s), and about 500 MB
of RAM.
- Faster TIE unpack by merging matrix groups, more efficient vertex
transforms, and skipping normal transforms on groups with no normals.
- Refactor generic merc and merc to use a single renderer with multiple
interfaces, rather than many renderers. Removed "LightningRenderer" as a
special thing, but Warp is still special
- Add more profiling stuff to startup and the loader.
- Remove `SDL_INIT_HAPTIC` - this turned out to be needed for
force-feedback steering wheels, and not needed for controller vibration
- Switched `vag-player` to use quicksort instead of the default GOAL
sort (very slow)
This moves the blerc math from mips2c to the Merc2 renderer, and uses
floats instead.
We could potentially do this on the GPU, which would be even faster, but
this isn't that slow in the first place.
Adds support for adding custom subtitles to Jak 2 audio. Comes with a
new editor for the new system and format. Compared to the Jak 1 system,
this is much simpler to make an editor for.
Comes with a few subtitles already made as an example.
Cutscenes are not officially supported but you can technically subtitle
those with editor, so please don't right now.
This new system supports multiple subtitles playing at once (even from a
single source!) and will smartly push the subtitles up if there's a
message already playing:
![image](https://github.com/open-goal/jak-project/assets/7569514/033e6374-a05a-4c31-b029-51868153a932)
![image](https://github.com/open-goal/jak-project/assets/7569514/5298aa6d-a183-446e-bdb6-61c4682df917)
Unlike in Jak 1, it will not hide the bottom HUD when subtitles are
active:
![image](https://github.com/open-goal/jak-project/assets/7569514/d466bfc0-55d0-4689-a6e1-b7784b9fff59)
Sadly this leaves us with not much space for the subtitle region (and
the subtitles are shrunk when the minimap is enabled) but when you have
guards and citizens talking all the time, hiding the HUD every time
anyone spoke would get really frustrating.
The subtitle speaker is also color-coded now, because I thought that
would be fun to do.
TODO:
- [x] proper cutscene support.
- [x] merge mode for cutscenes so we don't have to rewrite the script?
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Normally, when they allocate a VagCmd, they do a bunch of stuff to clear
all the status bits and reset things
in particular the InitVAGCmd function does a lot
![image](https://github.com/open-goal/jak-project/assets/48171810/9b355020-ad37-496c-9438-2f8d34f24e0a)
but for the stereo command, they do a lot less:
![image](https://github.com/open-goal/jak-project/assets/48171810/12a36712-0e68-4377-a6be-3bde82c2aa15)
Which means that the new_stereo_command can just have random status bits
left over from whatever the last user had.
we seem to end up in a state where byte21 is set, and this causes
everything else to be wrong and off-by-one dma transfer. My guess is
that the original game avoided this bug due to lucky timing that I don't
understand.
I think the fix of just clearing byte21 is ok because there's no way
that the old value of the byte is useful after the command is
repurposed.
This PR is a combination of
https://github.com/open-goal/jak-project/pull/2507 and some additional
changes to port Shadow VU1 to OpenGL. As far as I can tell, it's
working.
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Saves 16 bits and lets us align the `color_index` field properly.
This shouldn't improve or decrease performance by any noticeable amount
except maybe in really low end systems.
Adds sprite distort, fixes buggy sprite rendering in progress, adds
scissoring support (used in various scrolling menus) and a very basic
implementation of `blit-displays`. This is enough to make the fade
effect in the progress menu work, along with all the menus working
properly without needing to use the REPL. This does not make screen
flipping and the filter when failing a mission work.
Added support in the decompiler for detecting `dma-buffer-add-gs-set`
and `dma-buffer-add-gs-set-flusha` and updated all of the Jak 2 code to
use it. Readability improved!
Fixes decompiler issue with `with-dma-buffer-add-bucket` not inlining
forms which broke syntax. Fixes store error warnings showing up for
non-existent stores, there is now a dedicated pass for this at the end.
I started work on making `BITBLTBUF` stuff work in the DirectRenderer,
but stopped for now because it wasn't strictly necessary. It will still
assert like before.
Slight change to float divide operations (again). Now it only turns into
inverse multiplication if the float is a power of 2 (positive or
negative). Non-zero immediate divisors will be compiled as regular float
divisions but will forgo the extra branches and checks for divide by
zero.
Also fixes#2584
Allows for removing a line in the subtitle editor finally, also fixed
the following:
- The path the editor used didn't include the game name
- Loading the subtitle project is too slow to do on startup now
(4-5seconds in a debug build), do it on demand now
Updates the decompiler for the new format and there's new macros. This
new format should be easier to read/parse.
Also rewrote `sp-init-fields!` (both jak 1 and 2) from assembly to GOAL.
Hopefully I did not miss any regressions in Jak 1/2 while updating the
files, it's a lot.
Adds a decent way to customize the folders the project file expects the
iso data and decompiler data to be in. When you run any version other
than the default, for example Jak 1 PAL, it uses the `gameName`
decompiler config to consume and output it's results.
However the project file will assume `jak1` unless you hard-code it
differently -- basically, it needs to be explicitly told just the
decompiler is told what version to use.
We now have a per-user REPL Config json file, so that can be used to
override the default `jak1` behaviour.
Fixes#1993
- decompile `neon-baron-part`, which also has the hideout door for some
reason
- improve a few error messages in static data decompilation
- fix bug with disabling fog in merc
- better handling of the `disable-fog` settings for merc, should fix the
spotlights. There's a setting in the merc effect, and also a runtime
flag for the draw-control. I'm not actually sure what reads these, but
the draw-control one is definitely used to disable fog on the
spotlights.
- increase merc draw limit to try to fix the issue about partially drawn
citizens in the city
- remove useless debug prints (it's okay to die in init, and the medium
load buffer size mode is understood now)
Definitely needs a clean up pass, but I think the functionality is very
close.
There's a few "hacks" still:
- I am using the emerc logic for environment mapping, which doesn't care
about the length of the normals. I can't figure out how the normal
scaling worked in etie. I want to do a little bit more experimentation
with this before merging.
- There is some part about adgifs for TIE and ETIE that I don't
understand. The clearly correct behavior of TIE/ETIE is that the alpha
settings from the adgif shader are overwritten by the settings from the
renderer. But I can't figure out how this happens in all cases.
- Fade out is completely disabled. I think this is fine because the
performance difference isn't bad. But if you are comparing screenshots
with PCSX2, it will make things look a tiny bit different.
My mistake -- testing focused too much on preserving the existing
behaviour I clearly forgot to make sure the new stuff worked properly.
Just had to early out and not modify the args if they were in the new
format.
An attempt to cleanup the last CLI interface we have left to cleanup.
- `gk` args now follow the typical convention ie. `--proj-path` instead
of `-proj-path`.
- args that are passed through to the rest of the application / the
game's runtime use the typical convention of following a `--`
- I'm thinking some args shouldn't be handled at this level ie
(`-nodisplay`, `-vm`, `-novm` or `-jak2`) These could be better
documented as legitimate flags and passed in via a nice struct. They
don't seem to be used in `InitParams` but I'll triple check.
There's a temporary shim here so there is no coupled release with the
launcher (right now it executes `gk` with a few args). So I just change
the old args into the new format. After one release cycle, I can change
it in the launcher and delete it here.
I am unsure if this will break the bash shellscript usages -- not sure
which args were usually passed into `$@`
![image](https://user-images.githubusercontent.com/13153231/222035309-b6601719-cdc9-40ee-b36e-e4b135d3f128.png)
Three main changes:
- Adds support for the texture scrolling effect used on conveyor belts,
and turn it on for jak 2.
- Use merc instead of generic in jak 1 for ripple/water/texscroll stuff
(non-ocean water, lava, dark eco, etc). This is a pretty big speedup in
a lot of places.
- Fix a really old bug with blending mode used to draw environment maps.
The effect is that envmaps were half as bright as they should have been.
As usual, there's a flag to go back to the old behavior on jak 1. Set
these to `#t` to use generic like we used to.
```
*texscroll-force-generic*
*ripple-force-generic*
```
The format has changed, and everything must be rebuilt (C++, FR3's, GOAL
code)
Support rendering eyes with merc for both jak 1 and jak 2.
For jak 1, everything should look the same, but merc will be used to
draw eyes. This means that jak is now fully drawn with merc!
For jak 2, eyes should draw, but there are still a few issues:
- the tbp/clut ptr trick is used a lot for these eye textures, so
there's a lot that use the wrong texture
- I had to enable a bunch more "texture uploads" (basically emulating
the ps2 texture system) in order to get the eyes to upload. It would be
much better if the eye renderer could somehow grab the texture from the
merc model info, skipping the vram addressing stuff entirely. I plan to
return to this.
- I disabled some sky draws in `sky-tng`. After turning on pris2
uploads, the sky flashed in a really annoying way.
Where applicable, of course.
My system language is set to English so I actually can't test this. If
anyone has their Windows language (NOT LOCALE) set to Spanish, German,
French, Italian or Japanese please test this.
Fixes#1734
Also fixes the opengoal debugger on Windows and fixes the decomp for
`menu` which was causing some crashes related to input handling.
Increase level heaps and borrow heaps. The level heap increase was
likely not needed, but better safe than sorry. We allocate the 128 MB
main heap anyway so there's no harm.
Also fix the crash when using `-boot`. As I thought it was just a
one-line typo in the kernel.
Adds the `pckernel` system to Jak 2, allowing you to do the PC-specific
things that Jak 1 lets you do like change game resolution, etc.
In other to reduce the amount of code duplication for something that
we're gonna be changing a lot over time, I split it into a few more code
files. In this new system, `pckernel-h.gc`, `pckernel-common.gc`
(previously `pckernel.gc`) and `pc-debug-common.gc` are the files that
should be shared across all games (I hacked the Jak 2 project to pull
these files from the Jak 1 folder), while `pckernel-impl.gc`,
`pckernel.gc` and `pc-debug-methods.gc` are their respective
game-specific counterparts that should be loaded after. I'm not fully
happy with this, I think it's slightly messy, but it cleanly separates
code that should be game-specific and not accidentally copied around and
code that should be the same for all games anyway.
This automatically generates documentation from goal_src docstrings,
think doxygen/java-docs/rust docs/etc. It mostly supports everything
already, but here are the following things that aren't yet complete:
- file descriptions
- high-level documentation to go along with this (think pure markdown
docs describing overall systems that would be co-located in goal_src for
organizational purposes)
- enums
- states
- std-lib functions (all have empty strings right now for docs anyway)
The job of the new `gen-docs` function is solely to generate a bunch of
JSON data which should give you everything you need to generate some
decent documentation (outputting markdown/html/pdf/etc). It is not it's
responsibility to do that nice formatting -- this is by design to
intentionally delegate that responsibility elsewhere. Side-note, this is
about 12-15MB of minified json for jak 2 so far :)
In our normal "goal_src has changed" action -- we will generate this
data, and the website can download it -- use the information to generate
the documentation at build time -- and it will be included in the site.
Likewise, if we wanted to include docs along with releases for offline
viewing, we could do so in a similar fashion (just write a formatting
script to generate said documentation).
Lastly this work somewhat paves the way for doing more interesting
things in the LSP like:
- whats the docstring for this symbol?
- autocompleting function arguments
- type checking function arguments
- where is this symbol defined?
- etc
Fixes#2215
This will probably take a while, since we also have to translate all the
text of the base game - Naughty Dog never translated this game to
Hungarian. This PR will stay a draft until it is complete.
We realized that every letter in our alphabet was already working, apart
from two: Ő and Ű (they are unique sounds, so leaving their marks
wouldn't be okay).
Since I did not find that double accent thing in the jak font, I decided
to use ~ (see my change in FontUtils.cpp). It is good enough, and my
memory tells me that I already saw this exact same "solution"
(workaround) somewhere in the past. If anyone knows a better solution,
please let us know.
We chose ID 14 for the Hungarian language, as it was the lowest free ID.
**Progress tracker**
A tick here means that everything was translated. It does not mean that
everything is perfect yet. We will review everything multiple times, to
have the best translations possible.
Game text:
- [x] base game text
- [x] pc port text
- [ ] credits text ?
[Sziloyoo](https://github.com/Sziloyoo) helped with reviewing my
changes, and gave advice/suggestions for some complicated translations.
Subtitles will be done in the future, not in this PR.
This PR adds a feature to merc2 to update vertices. This will be needed
to efficient do effects like blerc/ripple/texture scroll. It's enabled
for blerc in jak 1 and jak 2, but with a few disclaimers:
- currently we still use the mips2c blerc implementation, which is slow
and has some "jittering" because of integer precision. When porting to
PC, there was an additional synchronization problem because blerc
overwrites the merc data as its being read by the renderers. I _think_
this wasn't an issue on PS2 because the blerc dma is higher priority
than the VIF1 DMA, but I'm not certain. Either way, I had to add a mutex
for this on PC to avoid very slight flickering/gaps. This isn't ideal
for performance, but still beats generic by a significant amount in
every place I tested. If you see merc taking 2ms to draw, it is likely
because it is stuck waiting on blerc to finish. This will go away once
blerc itself is ported to C++.
- in jak 1, we end up using generic in some cases where we could use
merc. In particular maia in village3 hut. This will be fixed later once
we can use merc in more places. I don't want to mess with the
merc/generic selection logic when we're hopefully going to get rid of it
soon.
- There is no support for ripple or texture scroll. These use generic on
jak 1, and remain broken on jak 2.
- Like with `emerc`, jak 1 has a toggle to go back to the old behavior
`*blerc-hack*`.
- In most cases, toggling this causes no visual differences. One
exception is Gol's teeth. I believe this is caused by texture coordinate
rounding issues, where generic has an additional float -> int -> float
compared to PC merc. It is very hard to notice so I'm not going to worry
about it.
This adds environment mapping support to `Merc2`, and turns it on for
Jak 1 and Jak 2.
- The performance is much better
- Jak 1 can be toggled back to the old behavior with `(set! *emerc-hack*
#f)`. The new environment mapping is identical to the old one everywhere
I checked.
- Jak 1 still falls back to generic for ripple/texscroll/blerc/eyes -
there's still no dynamic texture or vertex updating support. The eye
detection stuff will sometimes flag stuff as eyes which is not eyes,
which is fine, but means that generic will be used in some places where
emerc could be used. For example, the shiny plates on jak's arm will be
drawn with generic because jak has eyes.
- Jak 2 hasn't been checked super carefully against PCSX2 yet.
- Jak 2 still isn't technically using emerc, but instead putting emerc
models in the merc bucket.
- The interface to merc is a lot different now and totally custom
OpenGOAL DMA code. The original merc drawing asm doesn't run anymore.
- The FR3 format changed
- Something funky going on with foreground lighting in escape, but
doesn't seem to be related to this change?
Performance comparison, jak 1, in likely the most generic-merc heavy
spot:
![image](https://user-images.githubusercontent.com/48171810/213882718-feb2ab59-95a9-44a2-b0e5-95fba860c7b0.png)
![image](https://user-images.githubusercontent.com/48171810/213882736-8dbbf4c9-6bbf-4d0b-96ce-78d63274660c.png)
Adding support for better child-type method docstrings. This is a
problem unique to methods.
Usually, a child-type will have the same signature and a common name
will apply, but the implementation is different. This means, you
probably want a different docstring to describe what is happening.
Currently this is possible to do via `:replace`. The problem with
replace is two fold:
- a replaced method ends up in the generated `deftype`...because you
usually change the signature!
- we don't put docstrings in the `deftype` in normal GOAL, this is just
something we do for the `all-types` file (they go in the `defmethod`
instead)
- more importantly, this means anytime you now want to change the
parent's name/args/return type -- you have to apply that change
everywhere.
So this is a better design you can now just declare the method like so:
```clj
(:override-doc "my new docstring" <method_id>)
```
And internally a pseudo-replaced method will be added, but it will
inherit everything from the parent (except the docstring of course)
Unrelated - I also made all the keyword args for declaring methods not
depend on ordering
This also adds support for documenting virtual and non-virtual state
handlers. For example:
```clj
(:states
(part-tester-idle (:event "test") symbol))
```
or
```clj
(idle () _type_ :state (:event "test") 20)
```
I will probably add the ability to give some sort of over-view docstring
at a later date.
Co-authored-by: water <awaterford111445@gmail.com>
- started documenting the files I glossed over, some are totally done,
others are just partially done
- I changed the decompiler to automatically initialize the
art-group-info from the json file. This makes updating gsrc, even a
single file at a time, have consistent naming
- Though I disabled this functionality for jak 1, as I have no idea if
using the ntsc art groups will cause a regression for different versions
- fix indentation for docstrings -- it still doesn't look great, but
this is now a formatting concern, rather than the docstring having a
bunch of happen-stance leading whitespace.
- make sure bsp is processed on `l` levels in extraction (caused missing
remaps)
- clean up a few prints in extraction
- handle the <15 byte differences in art group files automatically (no
more errors about file naming)
- fix potential exception thrown by merc2 in a few ways: fixed bad data
in FR3's, check texture index just in case, and handle exceptions a
little bit better (still a crash, but at least you get a print)
- fix mips2 ocean stuff causing ocean far crashes
Effects the following files:
- [x] vehicle-rider
- [x] vehicle-control
- [x] vehicle-effects
- [x] vehicle
~~- [ ] vehicle-util~~
- [x] vehicle-physics
- [x] vehicle-states
~~- [ ] vehicle-guard~~
~~- [ ] traffic-engine~~
~~- [ ] traffic-manager~~
With the exception of traffic-engine, most of these files are either
done or have 1-3 stubborn functions remaining. Draft while I try to
resolve as many as possible / cleanup names and such.
Co-authored-by: water <awaterford111445@gmail.com>
- lets you split up your `startup.gc` file into two sections
- one that runs on initial startup / reloads
- the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do
Co-authored-by: water <awaterford111445@gmail.com>
This adds a new ImGUI menu to help filter out the clutter on screen.
https://user-images.githubusercontent.com/13153231/210192912-b1c28319-bacb-449c-ad7f-e7308fb75f50.mp4
This also:
- moves the imgui display bool into a game specific config file (you can
hide it in jak1, and not in jak2)
- the config file also persists the settings from this menu (except the
filters for now, future TODO)
- there is a new `ignore_imgui_hide_keybind` in this file to ignore
hiding it when you press Alt
A common thing that can be forgotten about / confusing to new people is
that:
1. if you add an object to the `allowed_objs` list but it's also in the
`banned_objs` list -- the ban still takes precedence with no failure
2. if you add an object to the `allowed_objs` list but have not allowed
the DGO in `inputs.jsonc` it will also silently log a failure and
continue.
This PR turns both situations into an explicit error with advice/a
reminder on what to do to fix it.
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
This solves two main problems:
- the looming threat of running out of memory since every thread would
consume duplicate (and probably not needed) resources
- though I will point out, jak 2's offline tests seem to hardly use any
memory even with 400+ files, duplicated across many threads. Where as
jak 1 does indeed use tons more memory. So I think there is something
going on besides just the source files
- condense the output so it's much easier to see what is happening / how
close the test is to completing.
- one annoying thing about the multiple thread change was errors were
typically buried far in the middle of the output, this fixes that
- refactors the offline test code in general to be a lot more modular
The pretty printing is not enabled by default, run with `-p` or
`--pretty-print` if you want to use it
https://user-images.githubusercontent.com/13153231/205513212-a65c20d4-ce36-44f6-826a-cd475505dbf9.mp4
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.
![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)
As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)
This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
"numConnectToTargetAttempts": 1,
"jak2": {
"asmFileSearchDirs": [
"goal_src/jak2"
]
}
}
```
This also provides a way to make game-specific configurations for the
REPL fairly easily.
- You can define a `startup.gc` in your user folder, each line will be
executed on startup (deprecates the usefulness of some cli flags)
- You can define a `repl-config.json` file to override REPL settings.
Long-term this is a better approach than a bunch of CLI flags as well
- Via this, you can override the amount of time the repl will attempt to
listen for the target
- At the same time, I think i may have found why on Windows it can
sometimes take forever to timeout when the game dies, will dig into this
later
- Added some keybinds for common operations, shown here
https://user-images.githubusercontent.com/13153231/202890278-1ff2bb06-dddf-4bde-9178-aa0883799167.mp4
> builds the game, connects to it, attaches a debugger and continues,
launches it, gets the backtrace, stops the target -- all with only
keybinds.
If you want these keybinds to work inside VSCode's integrated terminal,
you need to add the following to your settings file
```json
"terminal.integrated.commandsToSkipShell": [
"-workbench.action.quickOpen",
"-workbench.action.quickOpenView"
]
```
The offline-tests are going to end up taking too long for jak 2, I did
some rough math and by the end of it we'll be spending almost 2 minutes
for a full offline test on my machine.
These changes allow us to throw hardware at the problem
Still some work to do to make the output nicer, but seems to be fairly
reliable. By default it still uses 1 thread, use `num_threads` to change
this.