Commit graph

58 commits

Author SHA1 Message Date
Tyler Wilding 431508aab1
cmake: handle the scenario of a commit having more than 1 tag (#3662)
Some checks failed
Build / 🖥️ Windows (push) Has been cancelled
Build / 🐧 Linux (push) Has been cancelled
Build / 🍎 MacOS (push) Has been cancelled
Lint / 📝 Formatting (push) Has been cancelled
Lint / 📝 Required Checks (push) Has been cancelled
Lint / 📝 Optional Checks (push) Has been cancelled
2024-09-09 23:49:03 -04:00
Tyler Wilding eb703ee96e
REPL related improvements and fixes (#3545)
Motivated by - https://github.com/open-goal/opengoal-vscode/pull/358

This addresses the following:
- Fixes #2939 spam edge-case
- Stop picking a different nREPL port based on the game mode by default,
this causes friction for tools in the average usecase (having a REPL
open for a single game, and wanting to connect to it). `goalc` spins up
fine even if the port is already bound to.
- For people that need/want this behaviour, adding per-game
configuration to the `repl-config.json` is on my todo list.
- Allows `goalc` to permit redefining symbols, including functions. This
is defaulted to off via the `repl-config.json` but it allows you to for
example, change the definition of a function without having to restart
and rebuild the entire game.
![Screenshot 2024-06-02
124558](https://github.com/open-goal/jak-project/assets/13153231/28f81f6e-b7b8-4172-9787-f96e4ab1305b)
- Updates the welcome message to include a bunch of useful metadata
up-front. Cleaned up all the startup logs that appear when starting
goalc, many of whom's information is now included in the welcome
message.
  - Before:

![image](https://github.com/open-goal/jak-project/assets/13153231/814c2374-4808-408e-9ed6-67114902a1d9)

  - After:
![Screenshot 2024-06-01
235954](https://github.com/open-goal/jak-project/assets/13153231/f3f459fb-2cbb-46ba-a90f-318243d4b3b3)
2024-06-03 00:14:52 -04:00
Hat Kid 62ef9fe49d
[wip] build actor tool (#3266)
This does a couple of things:

- The `custom_levels` folder was renamed to `custom_assets` and contains
`levels`, `models` and `texture_replacements` folders for Jak 1, 2 and 3
in order to keep everything regarding custom stuff in one place.
- With this, texture replacements now use separate folders for all games
- A build actor tool was added that generates art groups for custom
actors
- Custom levels can now specify what custom models from the `models`
folder they want to import, this will add them to the level's FR3.
- A `test-zone-obs.gc` file was added, containing a `test-actor` process
that uses a custom model as an example.

The build actor tool is still very WIP, the joints and the default
animation are hardcoded, but it allows for importing any GLB file as a
merc model.
2024-05-18 18:18:25 +02:00
Tyler Wilding 0a15ac1669
goalc: Fix new symbol trie's performance inefficiencies (#3443)
I believe this brings things back in line to where it was before:
Here are the first handful of files before the changes:
```
0.014 | gcommon.gc
0.006 | gkernel-h.gc
0.025 | gkernel.gc
0.002 | pskernel.gc
0.01 | gstring.gc
0.004 | gstate.gc
0.001 | kernel.gd
0.001 | types-h.gc
0.002 | vu1-macros.gc
0.003 | math.gc
0.01 | vector-h.gc
0.001 | gravity-h.gc
0.001 | bounding-box-h.gc
0.001 | matrix-h.gc
0.001 | quaternion-h.gc
0.001 | euler-h.gc
```
> first compile
```
0.161 | gcommon.gc
0.126 | gkernel-h.gc
0.174 | gkernel.gc
0.046 | pskernel.gc
0.08 | gstring.gc
0.048 | gstate.gc
0.001 | kernel.gd
0.052 | types-h.gc
0.009 | vu1-macros.gc
0.059 | math.gc
0.228 | vector-h.gc
0.026 | gravity-h.gc
0.006 | bounding-box-h.gc
0.002 | matrix-h.gc
0.028 | quaternion-h.gc
0.026 | euler-h.gc
```
> make a change in gcommon and recompile

With the changes:
```
0.015 | gcommon.gc
0.018 | gkernel-h.gc
0.039 | gkernel.gc
0.006 | pskernel.gc
0.015 | gstring.gc
0.009 | gstate.gc
0.005 | kernel.gd
0.006 | types-h.gc
0.006 | vu1-macros.gc
0.008 | math.gc
0.017 | vector-h.gc
0.004 | gravity-h.gc
0.004 | bounding-box-h.gc
0.005 | matrix-h.gc
0.005 | quaternion-h.gc
0.003 | euler-h.gc
```
> First compile, no difference expected

```
0.016 | gcommon.gc
0.008 | gkernel-h.gc
0.023 | gkernel.gc
0.002 | pskernel.gc
0.01 | gstring.gc
0.043 | gstate.gc
0.001 | kernel.gd
0.002 | types-h.gc
0.002 | vu1-macros.gc
0.003 | math.gc
0.013 | vector-h.gc
0.001 | gravity-h.gc
0.002 | bounding-box-h.gc
0.002 | matrix-h.gc
0.001 | quaternion-h.gc
0.001 | euler-h.gc
```
> Compile times seem to be back within margin of error -- some are
faster than the first compilation time.
2024-04-01 18:56:55 -04:00
Tyler Wilding 53277a65ad
LSP: A bunch of new OpenGOAL language features (#3437)
- Integrate the AST into the LSP, this makes parsing and tokenizing the
files much easier
- Consolidate most of the symbol info tracking in `goalc` to a single
map. Fixed some issues where the old map would never evict symbols when
re-compiling files. There is still some more to cleanup, but this now
can be used as an incrementally updated source-of-truth for the LSP
- re-compile files when they are saved. Ideally this would be done
everytime they are changed but that:
  - may be too aggressive
- goalc doesn't compile incrementally yet so it likely would be a worse
UX

Features added, see
https://github.com/open-goal/opengoal-vscode/issues/256
- Hover

![image](https://github.com/open-goal/jak-project/assets/13153231/58dadb5d-582c-4c1f-9ffe-eaa4c85a0255)

![image](https://github.com/open-goal/jak-project/assets/13153231/b383adde-57fc-462c-a256-b2de5c30ca9a)
- LSP Status fixed
- Type Hierarchy

![image](https://github.com/open-goal/jak-project/assets/13153231/8e681377-1d4e-4336-ad70-1695a4607340)
- Document Color

![image](https://github.com/open-goal/jak-project/assets/13153231/4e48ccd8-0ed1-4459-a133-5277561e4201)
- Document Symbols
![Screenshot 2024-03-27
004105](https://github.com/open-goal/jak-project/assets/13153231/8e655034-43c4-4261-b6e0-85de00cbfc7f)
- Completions
![Screenshot 2024-03-30
004504](https://github.com/open-goal/jak-project/assets/13153231/d123a187-af90-466b-9eb7-561b2ee97cd1)

---------

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
2024-03-30 19:49:07 -04:00
Tyler Wilding a7cf66fda6
formatter: support formatting bindings, for example in a let (#2883) 2023-08-05 15:23:09 -04:00
water111 6f244b11ef
[jak2] Work-in-progress texture animations (#2819) 2023-07-14 18:17:54 -04:00
Tyler Wilding c87db7e670
i18n: subtitle code cleanup and update new subtitle JSON files to be compatible with Crowdin (#2802)
The main thing that was done here was to slightly modify the new
subtitle-v2 JSON schema to be more similar to the existing one so that
it can properly be used in Crowdin.

Draft while I double-check the diff myself

Along the way the following was also done (among other things):
- got rid of as much duplication as was feasible in the serialization
and editor code
- separated the text serialization code from the subtitle code for
better organization
- simplified "base language" in the editor. The new subtitle format has
built-in support for defining a base language so the editor doesn't have
to be used as a crutch. Also, cutscenes only defined in the base come
first in the list now as that is generally the order you'd work from
(what you havn't done first)
- got rid of the GOAL subtitle format code completely
- switched jak 2 text translations to the JSON format as well
- found a few mistakes in the jak 1 subtitle metadata files
- added a couple minor features to the editor
- consolidate and removed complexity, ie. recently all jak 1 hints were
forced to the `named` type, so I got rid of the two types as there isn't
a need anymore.
- removed subtitle editor groups for jak 1, the only reason they existed
was so when the GOAL file was manually written out they were somewhat
organized, the editor has a decent filter control, there's no need for
them.
- removed the GOAL -> JSON python script helper, it's been a month or so
and no one has come forward with existing translations that they need
help with migrating. If they do need it, the script will be in the git
history.

I did some reasonably through testing in Jak1/Jak 2 and everything
seemed to work. But more testing is always a good idea.

---------

Co-authored-by: ManDude <7569514+ManDude@users.noreply.github.com>
2023-07-09 02:53:39 +01:00
Tyler Wilding 436bac83ec
game: Improve OpenGL version detection and make requirement errors more obvious to the user (#2787) 2023-06-30 21:05:58 -04:00
Tyler Wilding 10934f6746
d/j2: Some work on the SQL editors (#2771) 2023-06-25 16:51:46 -04:00
ManDude 18ddd1613c
Jak 2 pc subtitle support (#2672)
Adds support for adding custom subtitles to Jak 2 audio. Comes with a
new editor for the new system and format. Compared to the Jak 1 system,
this is much simpler to make an editor for.

Comes with a few subtitles already made as an example.
Cutscenes are not officially supported but you can technically subtitle
those with editor, so please don't right now.

This new system supports multiple subtitles playing at once (even from a
single source!) and will smartly push the subtitles up if there's a
message already playing:

![image](https://github.com/open-goal/jak-project/assets/7569514/033e6374-a05a-4c31-b029-51868153a932)

![image](https://github.com/open-goal/jak-project/assets/7569514/5298aa6d-a183-446e-bdb6-61c4682df917)

Unlike in Jak 1, it will not hide the bottom HUD when subtitles are
active:

![image](https://github.com/open-goal/jak-project/assets/7569514/d466bfc0-55d0-4689-a6e1-b7784b9fff59)

Sadly this leaves us with not much space for the subtitle region (and
the subtitles are shrunk when the minimap is enabled) but when you have
guards and citizens talking all the time, hiding the HUD every time
anyone spoke would get really frustrating.

The subtitle speaker is also color-coded now, because I thought that
would be fun to do.

TODO:
- [x] proper cutscene support.
- [x] merge mode for cutscenes so we don't have to rewrite the script?

---------

Co-authored-by: Hat Kid <6624576+Hat-Kid@users.noreply.github.com>
2023-06-08 01:04:16 +01:00
Tyler Wilding 6d620a9289
formatter: extract formatting into a rules abstraction, add inner formatting (body vs argument functions) (#2684) 2023-06-04 13:19:29 -04:00
Tyler Wilding 4c6982b0ec
formatter: initial and basic indentation/alignment and expose the formatting via the LSP (#2673) 2023-05-28 13:22:00 -04:00
Tyler Wilding 0ffb912a04
formatter: add tree-sitter dependency and commit early draft work on a proper code formatter (#2536) 2023-04-24 23:46:55 -04:00
Tyler Wilding a264b6539b
game: Remove temporary CLI arg shim in gk (#2532) 2023-04-22 14:13:57 -04:00
Tyler Wilding 00ac12094e
goalc/repl: cleanup of goalc/REPL code and some QoL improvements (#2104)
- lets you split up your `startup.gc` file into two sections
  - one that runs on initial startup / reloads
  - the other that runs when you listen to a target
- allows for customization of the keybinds added a month or so ago
- removes a useless flag (--startup-cmd) and marks others for
deprecation.
- added another help prompt that lists all the keybinds and what they do

Co-authored-by: water <awaterford111445@gmail.com>
2023-01-07 11:24:02 -05:00
Tyler Wilding 2f4146d469
tests: make the offline tests aware of the current terminals row count (#2105)
This fixes the hideous output when your terminal would be too small to
hold all the threads.
2023-01-07 10:35:12 -05:00
water111 73561f10a3
support c++ tools on macos (#2063)
Running reference tests/decompiler should now be possible on macos
(arm). Most of the changes were just cleaning up places where we were
sloppy with ifdefs, but there were two interesting ones:
- `Printer.cpp` was updated to not use a recursive function for printing
lists, to avoid stack overflow
- I replaced xxhash with another version of the same library that
supports arm (the one that comes in zstd). The interface is C instead of
C++ but it's not bad to use. I confirmed that the extractor succeeds on
jak 1 iso so it looks like this gives us the same results as the old
library.
2022-12-22 17:12:05 -05:00
Tyler Wilding ac3c4e59b0
goalc/repl: Allow hot-loading files via ml with just the object name (#2036)
This allows you to not have to define the entire file path to a source
file to re-compile and load it. Technically a stop-gap until editor
tools are developed around writing OpenGOAL.


![image](https://user-images.githubusercontent.com/13153231/203196148-de61cf4b-42c8-43dc-a7fd-80e6ba6f5ac2.png)

As opposed to `(ml "goal_src/jak2/engine/game/main.gc")` (which still
works)

This is accomplished via the following config (connection attempts is
irrelevant):
```json
{
  "numConnectToTargetAttempts": 1,
  "jak2": {
    "asmFileSearchDirs": [
      "goal_src/jak2"
    ]
  }
}
```

This also provides a way to make game-specific configurations for the
REPL fairly easily.
2022-11-29 19:22:22 -05:00
Tyler Wilding f6bdc07990
d/jak2: finish progress menu code and initialize the camera (#1945)
This PR does a few main things:
- finish decompiling the progress related code
- implemented changes necessary to load the text files end-to-end
   - japanese/korean character encodings were not added
- finish more camera code, which is required to spawn the progress menu
/ init the default language settings needed for text
  - initialized the camera as well

Still havn't opened the menu as there are a lot of checks around
`*target*` which I havn't yet gone through and attempted to comment out.
2022-10-11 18:30:26 -04:00
Tyler Wilding 1b67d1dc77
Fix UTF-8 handling when running the game, env-vars, and setting the project path (#1632)
* fix utf-8 handling around env-vars

* fix file opening errors related to unicode

* add uncaught exception handler in `gk` to ensure something is logged

* gracefully fail if window icon cant be loaded and work with unicode

* linux fix and add changes to vendor file
2022-07-10 19:03:24 -04:00
water111 28a2ecdfd3
[jak2] goalc supports multiple projects (#1619)
* [jak2] goalc supports multiple projects

* disable deci2 server if not debugging
2022-07-06 21:18:08 -04:00
Tyler Wilding 6446389263
extractor: cleanup, support unicode properly, and add multi-game support (#1609)
* extractor: refactor and cleanup for multi-game support

* deps: switch to `ghc::filesystem` as it is utf-8 everywhere by default

* extractor: finally working with unicode

* unicode: fix unicode cli args on windows in all `main` functions
2022-07-05 20:38:13 -04:00
ManDude 9676100039
finish out english subtitles (#1586)
* put some duplicated code in a func

* make jak 2 text "work"

* group up all subtitles c++ code into one folder

* compact single-line subtitles

* fix a couple compiler crashes

* Update game_subtitle_en.gd

* `rolling` and `sunken`

* `swamp`

* `ogre`

* `village3`

* `maincave`

* `snow`

* `lavatube`

* `citadel`

* Update .gitignore

* clang

* fix encoding and decoding for quote

* properly fix quotes

* subtitle deserialize: sort by kind, ID and name

* sub editor: fix line speaker not being converted

* cleanup game text ids 1

* update text ids 2

* update source

* update refs
2022-07-03 17:25:28 -04:00
water111 c13934708a
Initial implementation of custom level tool (#1482)
* wip

* learning about colors

* gltf node stuff working

* cleanup

* support textures

* bvh generation seems reasonable

* tree layout

* frag packer, untested and doesnt do real stripping yet

* temp

* working collide frags

* handle bad inputs better

* clean up

* format

* include

* another include

* reorganize for release build use
2022-06-19 20:44:07 -04:00
water111 196c09a232
Clean up libstb_image (#1494) 2022-06-19 19:48:34 -04:00
Tyler Wilding 9fdf0bbc2f
tools: Add cutscene player / subtitle editor window (#1429)
* stash

* temp

* tools: subtitle tool works! just gotta fill out the db / polish UX

* tools: added configuration for every subtitle we have so far

* tools: add some colors to the editor, time for repl controls and make it run the code!

* tools: continuing polish of tool, getting very close

* tools: finished UX polish, just need to write deserializers

* tools: added deserializer for subtitle data

* tools: exported subtitle files, all data appears intact

* tools: more UX polish and test all the cutscenes, majority work

* assets: update subtitle files

* lint: formatting and cleanup

* lint: codacy lints
2022-06-11 16:32:27 +01:00
Tyler Wilding 7b6d732a77
goalc: Add TCP server socket in REPL process (#1335)
* goalc: cleanup goalc's main method and add nrepl listener socket

* deps: add standalone ASIO for sockets

* lint: formatting

* common: make a common interface for creating a server socket

* goalc: setup new repl server

* deps: remove asio

* goalc: debug issues, nrepl is working again

* git: rename files

* attempt to fix linux function call

* test

* scripts: make the error message even more obvious....

* goalc: make suggested changes, still can't reconnect properly

* game: pull out single-client logic from XSocketServer

* nrepl: supports multiple clients and disconnection/reconnects

* goalc: some minor fixes for tests

* goalc: save repl history when the compiler reloads

* common: add include for linux networking

* a few small changes to fix tests

* is it the assert?

* change thread start order and add a print to an assert

Co-authored-by: water <awaterford111445@gmail.com>
2022-05-06 18:19:37 -04:00
water111 789c57916e
Add timeline style profiler (#1312)
* small speedups to extractor

* faster extraction

* add profiler

* spellin

* guess at windows includes

* windows nominmax garbage
2022-04-17 21:12:24 -04:00
water111 97dc0e14df
[decompiler] performance improvements in extraction (#1309)
* small speedups to extractor

* faster extraction
2022-04-15 20:40:10 -04:00
water111 60a490d5c3
support unpacking iso files in the extractor (#1300) 2022-04-12 20:15:30 -04:00
Tyler Wilding e74bd3b32b
ci: Add an automated release process (#1262)
* ci: bring over some of my code to enable the process

* ci: cleanup builds a bit and add release stuff

* ci: fix paths and such after debugging

* ci: fix flowchart

* cmake: easily toggle between building everything dyn or statically linked

* ci: build release artifacts statically linked

* ci: fix some issues after testing once again, linux binaries are still too big
2022-03-29 22:47:08 -04:00
water111 78cde74d5a
update readme and fix always playing str (#1139)
* update readme deps

* replace assert

* bump timeout

* fix memory corruption in kernel

* use unknown if level name is invalid
2022-02-08 19:02:47 -05:00
ManDude 5b44aece75
random fixes + support clang-cl on visual studio (#1129)
* delete unused shaders

* hide some options in debug menu

* change fullscreen logic a bit

* add "all actors" toggle

* borderless fix and fix alpha in direct renderer untextured (do we need a separate shader for that?)

* fix fuel cell orbit icons in widescreen

* fix `curve` types

* refs

* fix levitator task...

* fix some task stuff

* update font code a bit (temp)

* cmake, third-party and visual studio overhaul

* Update .gitmodules

* update modules

* clone repos

* fix encoding in zydis

* where did these come from

* try again

* add submodule

* Update 11zip

* Update 11zip

* Update 11zip

* delete

* try again

* clang

* update compiler flags

* delete 11zip. go away.

* Create memory-dump-p2s.py

* properly

* fix minimum architecture c++ compiler flags

* fix zydis

* oops

* Update all-types.gc

* fix clang-cl tests

* make "all actors" work better, entity debug qol

* update game-text conversion code to be more modularized

* Create vendor.txt

* fix typos and minor things

* update refs

* clang

* Attempt to add clang-cl support to vs2019 and CI

* vs2022 + clang-cl

* srsly? fix clang build

* Update launch.vs.json

* extend windows CI timer
2022-02-07 19:15:37 -05:00
water111 f142f46045
add more comments/docstrings on first 58 files, try new windows-specific framelimiting (#1134)
* cleanup

* type

* link against a library
2022-02-05 16:30:50 -05:00
water111 a8342aef31
[graphics] TIE extractor (#1026)
* temp

* temp

* wip

* more progress on the instance asm

* first half of tie extraction, up to dma lists

* more tie extraction

* first part figured out maybe

* bp1 loop seems to work, bp2 loop does not

* bp1 and bp2 appear working. sadly ip is needed

* ip1 outline, not working ip2

* just kidding, ip2 seems to work

* extraction seems to work

* basic rendering working

* tie fixes

* performance optimization of tie renderer

* hook up tie to engine

* fix more bugs

* cleanup and perf improvements

* fix tests

* ref tests

* mm256i for gcc

* CLANG

* windows

* more compile fixes

* fix fast time of day

* small fixes

* fix after merge

* clang
2021-12-26 12:33:51 -05:00
water111 dbc266c00b
New Pretty Printer (#994)
* begin work on improved pretty printer

* update ref

* finish pretty printer

* force line break for defstate
2021-12-04 16:06:01 -05:00
water111 083202929c
[graphics] tfrag3 renderer (#978)
* begin work

* work

* working objs

* exporting

* it works

* before some time of day fixes

* add time of day interp and also fix zbuffer

* some small blending fixes

* improve randomess

* clean up extraction and missing blend mode

* culling, time of day, more level fixes

* more cleanup

* cleanup memory usage

* windows fix
2021-12-04 12:33:18 -05:00
water111 380e7b3580
clean up a few small things (#968) 2021-11-14 12:45:47 -05:00
Tyler Wilding 44c43610ce
game: Get camera code working (#965)
* cmake: disable edit&continue flags

* goos: make the build system work for alternate file paths nicely

* vs: update vs config

* vscode: extend terminal buffer!

* vs: fix presets

* debugger: fix exception handler

* game: add logo to application

* decomp: get `cam-master` to "work" -- manually changed return type

* debugger: fix printing issue

* game: get the camera actually working

* game: neutralize the analog sticks

* game: support analog sticks

* tests: update ref tests

* temp commit - inprogress stuff

* fix `send-macro`

* turn camera stuff back on, seems to work.  Still kernel-dispatch problem though

* address feedback

* formatting
2021-11-13 22:41:15 -05:00
water111 6a606d7222
Start a debug tool for looking through level data (#954)
* temp

* temp

* level inspection tool

* docs

* windows fix
2021-10-31 13:12:50 -04:00
ManDude 0e96cdb6c5
[de/compiler] New text tool (#918)
* fix `citb-drop-plat` a bit and PAL `fisher`

* Clean up `pc-pad-utils`. Looks so clean!

* Increase process stacks by ~2x

* Convert `game_text` custom encoding to and from a readable one (UTF-8)

* clang

* add missing characters

* support all diacritic variants

* fix a character

* remaining cases

* fix tests

* fix memory leak?

* clang

* add custom characters w/ diacritics

* Update all-types.gc

* robustness

* minor bug

* move custom font decoding function to `FontUtils.cpp`

* Move valid source chars patching to Reader constructor
2021-10-19 00:02:18 -04:00
water111 3a1c9eaf75
[decompiler] clean up offline test program, reorganize decomp order (#895)
* make a new offline test

* finish up offline test and fix crash bugs
2021-10-12 20:33:26 -04:00
water111 d777337095
add zstd library (#854) 2021-09-21 22:52:18 -04:00
water111 95366d21df
Get started on collide-func and clean up log forms in decompiler (#713)
* wip

* temp

* temp2

* first part of log macros

* more log macros

* logtest

* clean up

* dont initialize game info because we are missing stuff
2021-07-23 20:51:26 -04:00
water111 9ed1170046
add some really old vag file unpacking stuff (#708) 2021-07-18 19:28:56 -04:00
water111 b70fcb2f7a
support non-virtual gos in decompiler (#707) 2021-07-17 18:07:21 -04:00
water111 a6258f3654
[compiler] support 128-bit bitfields (#500)
* support setting and accessing fields of a 128-bit bitfield

* remove print

* rework static constants

* support 128-bit bitfields as part of static structures

* dynamic construction
2021-05-18 21:25:29 -04:00
water111 60c670df3a
Use dragonbox to print floats (#481)
* add new float printer

* more includes

* compare as floats
2021-05-13 21:05:05 -04:00
ManDude 0560136f08
Allow enum names to serve as types (using the enum's type) (#372)
* Allow enum names to serve as types (using the enum's type)

* Formatting

* add doc to `get_enum_type_name`

* Ban enum forward-declaring, and remove unneeded checks
2021-04-19 20:29:38 -04:00