Mainly:
- Make `max_descriptors_per_pool` project setting Vulkan-specific.
- Use a common, render driver agnostic magic FourCC for shader binary data.
- Downgrade spirv_reflect to Vulkan-only dependency.
- Add a `RENDER_DRIVER_*` macro to GLSL shader code for per-driver customizations.
Implements https://github.com/godotengine/godot-proposals/issues/3371.
New `target` presets
====================
The `tools` option is removed and `target` changes to use three new presets,
which match the builds users are familiar with. These targets control the
default optimization level and enable editor-specific and debugging code:
- `editor`: Replaces `tools=yes target=release_debug`.
* Defines: `TOOLS_ENABLED`, `DEBUG_ENABLED`, `-O2`/`/O2`
- `template_debug`: Replaces `tools=no target=release_debug`.
* Defines: `DEBUG_ENABLED`, `-O2`/`/O2`
- `template_release`: Replaces `tools=no target=release`.
* Defines: `-O3`/`/O2`
New `dev_build` option
======================
The previous `target=debug` is now replaced by a separate `dev_build=yes`
option, which can be used in combination with either of the three targets,
and changes the following:
- `dev_build`: Defines `DEV_ENABLED`, disables optimization (`-O0`/`/0d`),
enables generating debug symbols, does not define `NDEBUG` so `assert()`
works in thirdparty libraries, adds a `.dev` suffix to the binary name.
Note: Unlike previously, `dev_build` defaults to off so that users who
compile Godot from source get an optimized and small build by default.
Engine contributors should now set `dev_build=yes` in their build scripts or
IDE configuration manually.
Changed binary names
====================
The name of generated binaries and object files are changed too, to follow
this format:
`godot.<platform>.<target>[.dev][.double].<arch>[.<extra_suffix>][.<ext>]`
For example:
- `godot.linuxbsd.editor.dev.arm64`
- `godot.windows.template_release.double.x86_64.mono.exe`
Be sure to update your links/scripts/IDE config accordingly.
More flexible `optimize` and `debug_symbols` options
====================================================
The optimization level and whether to generate debug symbols can be further
specified with the `optimize` and `debug_symbols` options. So the default
values listed above for the various `target` and `dev_build` combinations
are indicative and can be replaced when compiling, e.g.:
`scons p=linuxbsd target=template_debug dev_build=yes optimize=debug`
will make a "debug" export template with dev-only code enabled, `-Og`
optimization level for GCC/Clang, and debug symbols. Perfect for debugging
complex crashes at runtime in an exported project.
- `_DEBUG` is MSVC specific so it didn't make much sense to define for
Android and iOS builds.
- iOS was the only platform to define `DEBUG`. We don't use it anywhere
outside thirdparty code, which we usually don't intend to debug, so it
seems better to be consistent with other platforms.
- Consistently define `NDEBUG` to disable assert behavior in both `release`
and `release_debug` targets. This used to be set for `release` for all
platforms, and `release_debug` for Android and iOS only.
- Due to the above, I removed the only use we made of `assert()` in Godot
code, which was only implemented for Unix anyway, should have been
`DEV_ENABLED`, and is in PoolAllocator which we don't actually use.
- The denoise and recast modules keep defining `NDEBUG` even for the `debug`
target as we don't want OIDN and Embree asserting all over the place.
This makes it possible to retrieve all relevant versioning info used to
generate `core/version_generated.gen.h` in the buildsystem.
Notably it makes the custom logic parsing the `GODOT_VERSION_STATUS`
environment variable to override status easy to reuse.
Adds support for LTO on macOS and Android. We don't have much experience
with LTO on these platforms so for now we keep it disabled by default
even when `production=yes` is set.
Similarly for iOS where we ship object files for the user to link in
Xcode so LTO makes builds extremely slow to link.
`production=yes` defaults to full LTO.
ThinLTO is much faster for LLVM-based compilers but seems to produce
bigger binaries (at least for the Web platform).
Modules can now call:
env.module_add_dependencies(name: str, deps: list, optional: bool)
To add required or optional dependencies during the "can_build" step.
Required dependencies will be checked and the module will be not be
enabled when they are missing, printing a warning to notify the user.
Adds SCons options to disable Brotli and Graphite.
Adds option categories to the build profiles editor.
Adds options default state to the build profiles editor.
Adds Text Server related options to the build profiles editor.
Fix misplaced OpenGL/Vulkan SCons options.
This PR is a continuation of #50381 (which was implemented exactly a year ago!)
* Add a visual interface to select which classes should not be built into Godot (well, they are built if something else uses them, but if not used the optimizer will remove them out).
* Add a detection system to scan the project and figure out the actual classes used.
* Added the ability for SCons to load build profiles.
Obligatory Screen:
A simple test with a couple of nodes in the scene resulted in a 25% reduction for the final binary size
TODO:
* Script languages need to implement used class detection (left for another PR).
* Options to disable servers or server functionalities (like 2D or 3D physics, navigation, etc). Are missing, that should also greatly aid in reducing binary size.
* Options to disable some modules would be desired.
* More options to disable drivers (OpenGL, Vulkan, etc) would be desired.
In general this PR is a starting point for more contributors to improve and enhance this functionality.
This doesn't change the behavior when `--jobs`/`-j` is specified as a
command-line argument or in `SCONSFLAGS`.
The SCons hack used to know if `num_jobs` was set by the user is derived
from the MongoDB setup.
We use `os.cpu_count()` for portability (available since Python 3.4).
With 4 CPUs or less, we use the max. With more than 4 we use max - 1 to
preserve some bandwidth for the user's other programs.
We shouldn't presume that future compilers will not have false positives or
find new occurrences of this warning, which would break compiling old versions
of the engine without passing custom `CXXFLAGS`.
Follow-up to #60652.
SConstruct change also makes it possible to outright delete the `editor`
folder in a `tools=no` build, which we use in CI to ensure no invalid
cross-dependencies are added.
It has been disabled in `master` since one year (#45852) and our plan
is for Bullet, and possibly other thirdparty physics engines, to be
implemented via GDExtension so that they can be selected by the users
who need them.
This reverts #53828 which had caused a significant drop in incremental
rebuild time for debug builds (from 10s to 23s on my laptop).
The "faster but unsafe" options are re-added, as well as adding
`max_drift=60` which we didn't use previously.
These options speed up SCons' own processing of the codebase to decide
what to build/rebuild (i.e. the first step before actually calling the
compiler). This will therefore not make much difference for scratch
builds, and is mostly useful for incremental rebuilds (including "null"
rebuilds with no change).
These options are enabled automatically for `debug` builds, unless
`fast_unsafe=no` is passed.
They are disabled by default for `release` and `release_debug` builds,
unless `fast_unsafe=yes` is passed.
The Godot Project Leadership Committe has decided to update the sponsoring
tiers so that the Platinum Sponsorship no longer includes a logo on the
editor splash screen.
This lets us reclaim the editor splash screen space for community-related
content instead of sponsors (e.g. a different community-designed splash
screen for each stable branch?).
Also removes two Platinum Sponsors whose sponsorship has expired earlier this
year.
When disabling specific modules, misspellings can occur. Additionally,
when switching between the `3.x` and `master` branches frequently,
it's possible to forget about renamed modules such as `lightmapper_cpu`
versus `lightmapper_rd`.
Note, the editor build requires the mbedtls module to be manually
enabled, as it is currently needed as a ResourceUID dependency.
This will need to be addressed in a separate PR.
- Rename OpenGL to GLES3 in the source code per community feedback.
- The renderer is still exposed as "OpenGL 3" to the user.
- Hide renderer selection dropdown until OpenGL support is more mature.
- The renderer can still be changed in the Project Settings or using
the `--rendering-driver opengl` command line argument.
- Remove commented out exporter code.
- Remove some OpenGL/DisplayServer-related debugging prints.
First implementation with Linux display manager.
- Add single-threaded mode for EditorResourcePreview (needed for OpenGL).
Co-authored-by: clayjohn <claynjohn@gmail.com>
Co-authored-by: Fabio Alessandrelli <fabio.alessandrelli@gmail.com>
We've had many issues with WebM support and specifically the libvpx library
over the years, mostly due to its poor integration in Godot's buildsystem,
but without anyone really interested in improving this state.
With the new GDExtensions in Godot 4.0, we intend to move video decoding to
first-party extensions, and this would likely be done using something like
libvlc to expose more codecs.
Removing the `webm` module means we can remove libsimplewebm, libvpx and
opus, which we were only used for that purpose. Both libvpx and opus were
fairly complex pieces of the buildsystem, so this is a nice cleanup.
This also removes the compile-time dependency on `yasm`.
Fixes lots of compilation or non-working WebM issues which will be linked
in the PR.
Current SCons 4.2.0 still supports Python 3.5 but deprecated it, and support
will be removed in the next release.
It's also become needlessly restrictive to prevent ourselves from using Python
3.6 f-Strings, so it's time to up the requirement.
They haven't really helped save much time on incremental rebuilds, and they do
cause potential issues with build correctness (and possibly even one of the cause
for overly eager incremental rebuilds).
Emscripten is LLVM-based so we want to follow the same logic. But we can't just
put it as a match in `methods.using_clang()` as that would mess with the
compiler version detection logic used to restrict old GCC and Clang releases.
It's raised for us on many comparators implemented to be able to store a struct
in `Set` or `Map` (who rely on `operator<` internally). In the cases I reviewed
we don't actually care about the ordering and we use the struct's function
pointers as that's the only distinctive data available.
* This PR adds the ability to disable classes when building.
* For now it's only possible to do this via command like:
`scons disable_classes=RayCast2D,Area3D`
* Eventually, a proper UI will be implemented to create a build config file to do this at large scale, as well as detect what is used in the project.
Debug builds are considerably slower than release builds or even
release_debug builds. `target=debug` is still the default SCons
target option, so unsuspecting users may be compiling unoptimized
debug builds for their personal use.
See discussion in #46814. Now going with the safe option again (like in 3.2)
as it turns out that we can't rely on user environments on Windows, since each
shell has a different set of env variables (especially the ones necessary to
use MSVC).
SCons does its own magic when we don't pass it an `ENV` dictionary, so we
should preserve it and only add things in a second step.
Fixes this warning when compiling with MSVC using git-bash.exe:
```
Missing environment variable: WindowsSdkDir
```
Possibly fixes build issues when having both MinGW and MSVC installed and an
older SCons version.
This fixes a regression from #46774 where `env["ENV"]` would miss some
important env variables on Windows, such as `SystemRoot`, `PATHEXT`, etc.
To have those, we can either use the default `ENV` created by SCons, or
propagate the whole external environment.
Fixes#46790.
We constructed the SCons environment without taking any (shell) environment
variables into account, and then appended a few, but too late. This would
cause variables like `env[CXX]` not to be properly expanded to respect a
non-standard `PATH`.
With this fix, setting:
```
PATH=$GODOT_SDK/bin:$PATH
```
will now properly use `$GODOT_SDK/bin/gcc` if available over `/usr/bin/gcc`.
The `dev=yes` and `production=yes` options work as aliases to set a number of
options, while still aiming to allow overriding specific options if the user
wishes so. (E.g. `production=yes use_lto=no` should work to enable production
defaults *but* disable LTO.)
That wasn't working as `ARGUMENTS.get()` returns a string and not a boolean as
expected by `BoolVariable`, and this wasn't flagged as a bug... So added a
helper method using SCons' `BoolVariable._text2bool` to do the conversion
manually.
This is meant for users making custom builds to match the options used on
optimized, official builds.
This enables, on the platforms which support them:
- `use_static_cpp=yes` (portable binaries for Linux and Windows)
- `use_lto=yes` (link time optimizations - note: requires a lot of RAM!)
- `debug_symbols=no` (no debug symbols, smaller binaries)
Also abort when using MSVC with `production=yes`, as:
- It cannot optimize the GDScript VM like GCC or Clang do, leading to
significant performance drops.
- Its LTO support is unreliable, at least used to trigger crashes last
we tried it extensively.
All options can still be overridden if specified, and the `dev=yes` option
was changed to also support overrides.
Fixes a pre-existing bug that #44433 exposed.
It's pretty hacky, but we use `platform` in `env` both as an optional command
line option (instead it can be autodetected, or passed via the `p` alias, and
on Linux it might be overridden if you pass one of the convenience alias
values), and as the reference value for what platform we're building on.
Thus we override `env_base["platform"]` with the autodetected or validated
platform, but any call to `opts.Update(env_base)` overrides it with the
original command line option... causing e.g. #44448.
The proper fix would be to refactor all this so that we don't reuse
`env["platform"]` for platform detection (it could instead be e.g.
`env.platform` as a member variable which holds the validated value),
but for now I'm tapering over the immediate breakage.
Fixes#44448 and other breakages induced by #44433.
Otherwise we can get situations where platform-specific opts with the same name
can override each other depending on the order at which platforms are parsed,
as was the case with `use_static_cpp` in Linux/Windows.
Fixes#44304.
This also has the added benefit that the `scons --help` output will now only
include the options which are relevant for the selected (or detected) platform.
ptrcall is now also used to optimize calls in GDScript, on top of the existing
use by the GDNative and Mono modules.
It no longer makes sense to make it optional.
This adds `custom_modules_recursive` which allows to detect and collect
all nested C++ modules which may reside in any directory specified by
`custom_modules` option.
The detection logic is made to be more strict because `SCSub` may be
used for organizing hierarchical builds within a module itself, so the
existence of `register_types.h` and `config.py` is checked as well
(these are all required for a C++ module to be compiled by Godot).
For performance reasons, built-in modules are not checked recursively,
and there's no benefit of doing so in the first place.
It's now possible to specify a directory path pointing to a *single*
module, as it may contain nested modules which are detected recursively.
This code currently isn't compiled (and cannot compile).
We plan to re-add OpenGL ES-based renderer(s) in Godot 4.0 alongside Vulkan
(probably ES 3.0, possibly also a low-end ES 2.0), but the code will be quite
different so it's not relevant to keep this old Godot 3.2 code.
The `drivers/gles2` code from the `3.2` branch can be used as a reference for
a potential new implementation.
A new `env.Run` method is added which allows to control the verbosity
of builders output automatically depending on whether the "verbose"
option is set. It also allows to optionally run any SCons commands in a
subprocess using the existing `run_in_subprocess` method, unifying
the interface. `Action` objects wrap all builder functions to include a
short build message associated with any action.
Notably, this removes quite verbose output generated by `make_doc_header`
and `make_editor_icons_action` builders.
Until https://github.com/psf/black/pull/1328 makes it in a stable release,
we have to use the latest from Git.
Apply new style fixes done by latest black.
