types-h, vu1-macros, gsound-h, dma-h, video-h, vu1-user-h, profile-h
The `gsound-h` is very simple, but the rest have decent docs and all the
macros we had from jak 2.
So far, barely any differences! (there's a few in gsound-h)
Decompile `gcommon`. I adjusted the spacing of docstring comments, and
removed some spammy decompiler warning prints.
I also added some random notes I had on VU programs from jak1/jak2. They
are not polished, but I think it's still worth including since we'll
have to go through them again for jak 3.
- `vector-h`
- `gravity-h`
- `bounding-box-h`
- `matrix-h`
- `quaternion-h`
- `euler-h`
- `transform-h`
- `geometry-h`
- `trigonometry-h`
- `transformq-h`
- `bounding-box`
- `matrix`
- `matrix-compose`
- `transform`
- `quaternion`
- `euler`
- `trigonometry`
Not a whole lot of changes, just a couple of new functions and one new
file (`matrix-compose`).
In #3300, `per_object` in `ObjectFileDB::ir2_analyze_all_types()` was
changed to an `unordered_map`, which messed with the object file order
in `all-types`. This fixes the issue by going through each object file
in order and then looking up the type info from the map instead of
iterating through the map.
This also removes the method IDs from the old method comments and adds a
stub for the `new` method if there is an old method comment for it, in
order to stay consistent with the rest of the method list.
This sets up the C Kernel for Jak 3, and makes it possible to build and
load code built with `goalc --jak3`.
There's not too much interesting here, other than they switched to a
system where symbol IDs (unique numbers less than 2^14) are generated at
compile time, and those get included in the object file itself.
This is kind of annoying, since it means all tools that produce a GOAL
object file need to work together to assign unique symbol IDs. And since
the symbol IDs can't conflict, and are only a number between 0 and 2^14,
you can't just hash and hope for no collisions.
We work around this by ignoring the IDs and re-assigning our own. I
think this is very similar to what the C Kernel did on early builds of
Jak 3 which supported loading old format level files, which didn't have
the IDs included.
As far as I can tell, this shouldn't cause any problems. It defeats all
of their fancy tricks to save memory by not storing the symbol string,
but we don't care.
This sets out the bones of a Jak 3 build, many things are stubbed out,
guessed, or copied from Jak 2 but it should at least be good enough to:
run `task set-game-jak3`
launch the repl
run builds from the repl
build outputs themselves are untested but the build itself runs without
errors
---------
Co-authored-by: Tyler Wilding <xtvaser@gmail.com>
This adds support for using enums in lumps using the new lump types
`enum-int32` and `enum-uint32`. Also adds these other new lump types:
- `water-height` (takes 3 meter floats, an enum and another optional
meter float)
- `eco-info` (takes an enum and an int)
- `vector3m` (3 meter floats + `w` set to 1.0)
The logger used in `goalc` tries to print an already-formatted string
`message` using `fmt::print(message);` Usually this doesn't cause
problems, but if you try to print, for example, an exception that has
special characters (notably `{`) it will try to do another round of
formatting/replacements, despite not having any args to replace with,
which ends up throwing another exception. This is why errors when
parsing custom level JSON cause the REPL to exit.
I've hopefully identified all the various instances of this across the
codebase
Currently only tracks enemy kills, and how they were killed. There is
currently no menu for this, but I've already added most of the text for
it. Also did a bunch of misc decompilation fixes and renamed some
methods.
Fixes#3277Fixes#3278
Adds the opengoal cheats to the secrets menu. Only cheats that are fully
functional and unlockable are there right now, which is eight cheats.
This update will reset most Jak 2 settings.
Also fixes#3274 .
There are art groups that are present in multiple levels, and that means
that also their textures are present in multiple levels. With texture
replacement, currently we need to make replacements for all instances if
we want it replaced everywhere, but this is not ideal, especially when
you make changes to your replacement texture and now you have to put it
in each folder again.
I added a way to replace all instances of a texture, by letting
texture-replacer people put their replacements into an '_all' folder. I
set up the logic in such a way that if you have a replacement for the
texture in its corresponding folder, it will take priority over a
replacement that you placed into the '_all' folder.
I personally found this very useful for replacing guard textures. The
guards appear in a lot of levels. But ideally you want them to look the
same everywhere. And that is why I looked into this and made a PR.
Oh and I changed what is printed in the 'Replacing ' part because it was
printing the path to our replacement, which didn't look nicely when
several textures got replaced by the same replacement from the '_all'
folder. So now it will print the original texture's page and name, I
think this information is more useful anyway.
Switches the slime look up table to be a texture, since I guess intel
drivers are terrible and putting the array in the shader makes it
extremely slow.
Also, a few minor changes:
- removed art-groups from the test-zone levels since this causes the
compiler to re-decompile the game, and makes the launcher slower. (left
it in commented out)
- Switched `decompile_code` to false by default in jak 2, in case people
run the decompiler and don't want to wait forever
- Fixed build warnings
Major change to how `deftype` shows up in our code:
- the decompiler will no longer emit the `offset-assert`,
`method-count-assert`, `size-assert` and `flag-assert` parameters. There
are extremely few cases where having this in the decompiled code is
helpful, as the types there come from `all-types` which already has
those parameters. This also doesn't break type consistency because:
- the asserts aren't compared.
- the first step of the test uses `all-types`, which has the asserts,
which will throw an error if they're bad.
- the decompiler won't emit the `heap-base` parameter unless necessary
now.
- the decompiler will try its hardest to turn a fixed-offset field into
an `overlay-at` field. It falls back to the old offset if all else
fails.
- `overlay-at` now supports field "dereferencing" to specify the offset
that's within a field that's a structure, e.g.:
```lisp
(deftype foobar (structure)
((vec vector :inline)
(flags int32 :overlay-at (-> vec w))
)
)
```
in this structure, the offset of `flags` will be 12 because that is the
final offset of `vec`'s `w` field within this structure.
- **removed ID from all method declarations.** IDs are only ever
automatically assigned now. Fixes#3068.
- added an `:overlay` parameter to method declarations, in order to
declare a new method that goes on top of a previously-defined method.
Syntax is `:overlay <method-name>`. Please do not ever use this.
- added `state-methods` list parameter. This lets you quickly specify a
list of states to be put in the method table. Same syntax as the
`states` list parameter. The decompiler will try to put as many states
in this as it can without messing with the method ID order.
Also changes `defmethod` to make the first type definition (before the
arguments) optional. The type can now be inferred from the first
argument. Fixes#3093.
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Some files were in the `banned_objects` list and were thus excluded from
the `all_objs` file.
Also implements the `pexcw` instruction which is only used in `hfrag`
code.
Fixes missing collision geometry reported in
https://github.com/open-goal/jak-project/issues/3011
The issue happens when there are 256 polygons. In this case `num-polys`
is 0 (it's a u8). There are actual cases where there are 0 polygons, so
we have to do a more complicated check to get the real count. I should
have done this in the first place, but it seemed to work...
Started at 349,880,038 allocations and 42s
- Switched to making `Symbol` in GOOS be a "fixed type", just a wrapper
around a `const char*` pointing to the string in the symbol table. This
is a step toward making a lot of things better, but by itself not a huge
improvement. Some things may be worse due to more temp `std::string`
allocations, but one day all these can be removed. On linux it saved
allocations (347,685,429), and saved a second or two (41 s).
- cache `#t` and `#f` in interpreter, better lookup for special
forms/builtins (hashtable of pointers instead of strings, vector for the
small special form list). Dropped time to 38s.
- special-case in quasiquote when splicing is the last thing in a list.
Allocation dropped to 340,603,082
- custom hash table for environment lookups (lexical vars). Dropped to
36s and 314,637,194
- less allocation in `read_list` 311,613,616. Time about the same.
- `let` and `let*` in Interpreter.cpp 191,988,083, time down to 28s.
This sets up the extractor for jak 2. I was expecting that I'd have to
make some more significant changes to the decompiler/compiler path
stuff, but this was not the case!
The only real change is that you can now provide multiple ISO hashes for
an entry in `ISOMetadata`. This is needed for the two different NTSC
versions, which have the same configs, serials, and ELF hashes, but
slightly different contents.
I also didn't add the korean version because I don't have the info for
it.
---------
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
This change adds a few new features:
- Decompiler automatically knows the type of `find-parent-method` use in
jak 1 and jak2 when used in a method or virtual state handler.
- Decompiler inserts a call to `call-parent-method` or
`find-parent-state`
- Removed most casts related to these functions
There are still a few minor issues around this:
- There are still some casts needed when using `post` methods, as `post`
is just a `function`, and needs a cast to `(function none)` or similar.
It didn't seem easy to change the type of `post`, so I'm not going to
worry about it for this PR. It only shows up in like 3 places in jak 2.
(and 0 in jak 1)
- If "call the handler if it's not #f" logic should probably be another
macro.
Fixes#805
Adds controller LED features to Jak 2:
- progressive flickering denoting health
- copies tomb simon says puzzle colors
- unique colors for each gun
- orange color for being indax
- yellow color for being in mech
- purple color for being darkjak
- blue color for being in board
- red flash when wanted.
May add more features later?
Also did some minor clean-up on some types.
This renames the method object in `defmethod`s to `this` and adds
detection for the `set-time!` and `time-elapsed?` macros.
Definitely my biggest PR yet...
- state handlers that are not inlined lambdas have smarter type
checking, getting rid of 99.9% of the casts emitted (they were not
useful)
- art groups were not being properly linked to their "master" groups.
- `max` in `ja` in Jak 2 was not being detected.
Another huge PR...
Previously, `object` and `none` were both top-level types. This made
decompilation rather messy as they have no LCA and resulted in a lot of
variables coming out as type `none` which is very very wrong and
additionally there were plenty of casts to `object`. This changes it so
`none` becomes a child of `object` (it is still represented by
`NullType` which remains unusable in compilation).
This change makes `object` the sole top-level type, and the type that
can represent *any* GOAL object. I believe this matches the original
GOAL built-in type structure. A function that has a return type of
`object` can now return an integer or a `none` at the same time.
However, keep in mind that the return value of `(none)` is still
undefined, just as before. This also makes a cast to `object`
meaningless in 90% of the situations it showed up in (as every single
thing is already an `object`) and the decompiler will no longer emit
them. Casts to `none` are also reduced. Yay!
Additionally, state handlers also don't get the final `(none)` printed
out anymore. The return type of a state handler is completely
meaningless outside the event handler (which is return type `object`
anyway) so there are no limitations on what the last form needs to be. I
did this instead of making them return `object` to trick the decompiler
into not trying to output a variable to be used as a return value
(internally, in the decompiler they still have return type `none`, but
they have `object` elsewhere).
Fixes#1703Fixes#830Fixes#928
This PR adds detection of the `launch-particles` and `seconds-per-frame`
macros to the decompiler, removing a lot of bloat and hiding many
process register uses.
I also added `og:preserve-this` comments to as many manual patches and
comments as I could, which will soon be used in conjunction with CI to
hopefully catch any regressions in future big decomp update PRs.
I have some concerns about the `launch-particles` macro (more details in
`sparticle-launcher.gc`) , but thus far, I have not seen anything break
yet.
---------
Co-authored-by: water <awaterford111445@gmail.com>
This will create a folder like `decompiler_out/jak1/entities` and save a
JSON file per level with all the actors.
Also, it should hopefully make custom level building a little faster.
I think this is very likely to fix
https://github.com/open-goal/jak-project/issues/2970
We had somebody report a stacktrace from the debugger, and it was
immediately after calling `gen-perms`.
I found that `gen-perms` writes past the end of a stack array during
this mission, and at the same time as the reported freezes.
I was unable to recreate the original freeze after making this change.
This PR adds a frame rate option to the graphics menu for some of the
most common refresh rates.
Jak 2 has much better support for variable frame rates than Jak 1 out of
the box, but there are still some edge cases, most prominently the fact
that sprites are still limited to the 300 tick system, which is most
noticeable on glow sprites. For this, I abused the glow boost debug
setting to scale the glow based on the frame rate.
While testing, I noticed two other cases that I have also patched,
there's likely to be many more that are yet to be found, but aside from
that, the game is playable as normal.
https://github.com/open-goal/jak-project/assets/6624576/ad4db24f-cd27-4237-a155-0db7008160f3
There are potentially still some minor issues with the resulting files.
Some of them appear to have minor artifacts that playing through the
actual game do not -- but this is a much better starting point for
someone to iterate from if they are interested in improving things.
Export models with "wind". The levels with wind models are:
firecanyon (9), beach (5), village1 (7), lavatube (2).
Sometimes a single object is made up of multiple models - for example
the tree in sandover is actually several meshes.
Fixes#2167
Reduces test flakiness if ran on multiple threads and gets rid of a few
hundred files from the source tree
I believe this also makes #1434 irrelevant and it can be closed.
---------
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
Fix the bug described in
https://github.com/open-goal/jak-project/issues/2882 where some shrubs
are transparent when they shouldn't be. The problem was that we never
carefully looked at the settings in `gs-prim`, which has a bit to
enable/disable alpha blending entirely. Now it should be correct for
both jak 1 and jak 2. To see this change, you'll need to re-extract.
Also adds a setting to disable saving texture .pngs, to speed up
decompilation. I left it on for jak 1 (to avoid confusion for texture
swapping(, but off for jak 2 for now.
Rotates the log files with a timestamp instead of copying all files and
incrementing an integer. Increases the amount of info you have when
looking at user's log files (ie. when looking at all the files, the file
creation dates are accurate).
Also simplifies the API for setting the log file, and `gk` logs are now
game specific with `jak1` or `jak2`. Which should be useful going
forward.
Lastly, added a flag to all CLIs to disable ansi colors for people that
want to do so. Though at the same time, there is finally a workaround in
jenkins to fix ANSI colors in the truncated log view -- so I'm not sure
why anyone would want to get rid of the color information. You can even
setup text editors to display the color info making log parsing much
easier. Fixes#1917
---------
Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
The progress menu loads its icon textures from a .STR file that we were
previously ignoring.
This change:
- updates the decompiler so it can process a .STR file containing a
texture
- adds a feature to force an entire page to always be loaded in the PC
renderer by putting all textures in the GAME.FR3 file.
- regenerates the texture offset map file for jak 2 with these new
textures
For now, I've just put the icon textures in GAME.FR3. The downside is
that these will always stay on the GPU, using up VRAM even when they
aren't needed. But the entire GAME.FR3 file is under 3 MB so I think
it's ok.
- Add security wall animation
- Add waterfall animations
- Add lava animations
- Update layer values from the game to fix the security wall
- Remove leftover debug in `level.gc` that would break level-specific
animations on the second time you visited the level
- Optionally load animated slot textures to the pool so generic can use
them (fixes skull gems in UI)
Added framework to do texture animations entirely in C++. Currently only
works on relatively simple ones, and doesn't handle updating all
parameters - only the speeds.
Connected texture animations to merc and tfrag for skull gems, dark
bomb, and scrolling conveyors.
Cleaned up Tfragment/Tfrag3, which used to be two classes. This was one
of the first C++ renderers, so it had a weird design.
I havn't tested it yet, but I can almost guarantee that atleast `goalc`
will not work in the slightest!
But the project is atleast fully compiling. My hope is to start
translating some AVX to NEON next / get `goalc` working...eventually.
Fixes empty boxed arrays of strings breaking some decomp
(`ctywide-speech` and `race-info`).
Adds `decomp-as` tag to decompiler types so that the static data
decompiler can use macros like `meters` and `seconds` on fields that
aren't of type `meters` or `time-frame`.
Adds `override` tag to decompiler types which overrides the type of
field with that name. The type must be a child type of the original
field's type (or the same type, but why would you do this?).
Fixes the camera being offset for `drillmtn` after loading `palout`
once.
This is a huge refactor sadly.
This is a major deviation from the original game, which did not have any
way to access the nest after beating Metal Kor as the air train gets
hidden when he is beaten. This was mostly annoying because there are
precursor orbs in that level that you might miss.
This makes it so the air train can once again be used to enter the nest
even after beating Metal Kor. The rest of the level remains mostly the
same, except the Rift Ring does not spawn and an invisible wall is added
to the Metal Kor arena to prevent you from entering it as you are
normally unable to leave it anyway.
Trying to make up for some of the startup speed lost in the SDL
transition. This saves about 1s from start (from ~3s), and about 500 MB
of RAM.
- Faster TIE unpack by merging matrix groups, more efficient vertex
transforms, and skipping normal transforms on groups with no normals.
- Refactor generic merc and merc to use a single renderer with multiple
interfaces, rather than many renderers. Removed "LightningRenderer" as a
special thing, but Warp is still special
- Add more profiling stuff to startup and the loader.
- Remove `SDL_INIT_HAPTIC` - this turned out to be needed for
force-feedback steering wheels, and not needed for controller vibration
- Switched `vag-player` to use quicksort instead of the default GOAL
sort (very slow)
```lisp
(defenum collide-action
:type uint32
:bitfield #t
(solid 0) ;; used for solid things
(rider-plat-sticky 1) ;; used for platforms in rider/platform interactions
(rider-target 2) ;; used for target in rider/platform interactions
(edgegrab-active 3) ;; set/cleared when entering/exiting edgegrab states
(rider-plat 4) ;; used for platforms in rider/platform interactions
(unused 5) ;; totally unused?
(edgegrab-possible 6) ;; used when edge grab checks should be done
(edgegrab-cam 7) ;; set/cleared when entering/exiting edgegrab states
(swingpole-active 8) ;; set/cleared when entering/exiting swingpole states
(racer 9) ;; set/cleared when entering/exiting racer states
(attackable 10) ;; used for something to do with attacking/damaging
(attackable-unused 11) ;; seems to relate to attacking - set in several places but never tested for?
(snowball 12) ;; set/cleared when entering/exiting snowball states
(tube 13) ;; set/cleared when entering/exiting tube states
(flut 14) ;; set/cleared when entering/exiting flutflut states
(racer-grounded 15) ;; set/cleared when entering/exiting certain racer states w/ extra conditions
(racer-unused 16) ;; seems to relate to racer - never set, only cleared in one place?
)
```
This moves the blerc math from mips2c to the Merc2 renderer, and uses
floats instead.
We could potentially do this on the GPU, which would be even faster, but
this isn't that slow in the first place.
Adds support for adding custom subtitles to Jak 2 audio. Comes with a
new editor for the new system and format. Compared to the Jak 1 system,
this is much simpler to make an editor for.
Comes with a few subtitles already made as an example.
Cutscenes are not officially supported but you can technically subtitle
those with editor, so please don't right now.
This new system supports multiple subtitles playing at once (even from a
single source!) and will smartly push the subtitles up if there's a
message already playing:
Unlike in Jak 1, it will not hide the bottom HUD when subtitles are
active:
Sadly this leaves us with not much space for the subtitle region (and
the subtitles are shrunk when the minimap is enabled) but when you have
guards and citizens talking all the time, hiding the HUD every time
anyone spoke would get really frustrating.
The subtitle speaker is also color-coded now, because I thought that
would be fun to do.
TODO:
- [x] proper cutscene support.
- [x] merge mode for cutscenes so we don't have to rewrite the script?
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Gives proper names to almost every color. It is very apparent that some
colors are context-sensitive/made for a specific purpose, so those
colors were named after that purpose instead of a generic color name.
Normally, when they allocate a VagCmd, they do a bunch of stuff to clear
all the status bits and reset things
in particular the InitVAGCmd function does a lot
but for the stereo command, they do a lot less:
Which means that the new_stereo_command can just have random status bits
left over from whatever the last user had.
we seem to end up in a state where byte21 is set, and this causes
everything else to be wrong and off-by-one dma transfer. My guess is
that the original game avoided this bug due to lucky timing that I don't
understand.
I think the fix of just clearing byte21 is ok because there's no way
that the old value of the byte is useful after the command is
repurposed.
Also fixed an original game bug in `loader.gc` on a method that's called
quite often, though I have no clue what erroneous behavior it could have
even caused.
- elec gates now always render at max quality if you have PS2 lods
disabled. the original render distances are so low that the one in
caspad is impossible to see in normal gameplay.
- `fort-entry-gate-11` and `com-airlock-outer-13` are specifically
banned from the all actors hack because they are placed in a bad spot
and Naughty Dog did not program the airlocks very well.
- fixed NPC talk distance being bad for 1 frame.
- fix `sew-scare-grunt` erroneously keeping its spool anim active if you
killed the enemy.
This PR is a combination of
https://github.com/open-goal/jak-project/pull/2507 and some additional
changes to port Shadow VU1 to OpenGL. As far as I can tell, it's
working.
---------
Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
Adds sprite distort, fixes buggy sprite rendering in progress, adds
scissoring support (used in various scrolling menus) and a very basic
implementation of `blit-displays`. This is enough to make the fade
effect in the progress menu work, along with all the menus working
properly without needing to use the REPL. This does not make screen
flipping and the filter when failing a mission work.
Added support in the decompiler for detecting `dma-buffer-add-gs-set`
and `dma-buffer-add-gs-set-flusha` and updated all of the Jak 2 code to
use it. Readability improved!
Fixes decompiler issue with `with-dma-buffer-add-bucket` not inlining
forms which broke syntax. Fixes store error warnings showing up for
non-existent stores, there is now a dedicated pass for this at the end.
I started work on making `BITBLTBUF` stuff work in the DirectRenderer,
but stopped for now because it wasn't strictly necessary. It will still
assert like before.
Updates the decompiler for the new format and there's new macros. This
new format should be easier to read/parse.
Also rewrote `sp-init-fields!` (both jak 1 and 2) from assembly to GOAL.
Hopefully I did not miss any regressions in Jak 1/2 while updating the
files, it's a lot.
- [x] compare NTSC-K
- [x] compare NTSC-J
- [x] compare PAL
- [x] figure out version order
- [x] ~~write delta patch for spanish text~~ no need for now
Fixes#2530
