Allow easy "func overriding" of specific map build steps for customization

[azur-wolve]

A core rebuild of FuncGodot has been mentioned various times.

It looks like a good moment for propose this.

Would want to suggest to make it easy to allow "overriding" functions of the FuncGodot core for custom map building.

Ideally consists in that every step would be split into a function for separation of concerns and easily having custom versions of it for a project.

Using #97 as example, there would be a function called ie _apply_tool_textures() to make it easier to modify just that particular aspect instead of being just mixed with other parts of the map build process.

Not sure if would be better to turn those funcs into callable variables, allowing to assign them a custom function that is assigned to.

This way would be able to just change specific functions rather than extend FuncGodotMap and rearranging / recreating all the map build pipeline of functions just to change a particular step.
So only assign a custom callable to FuncGodotMap.whatever_callable = custom_callable.

Not sure if that makes sense.

That would allow to have custom map build solution for features that are tied to projects or personal workflows that aren't merged into vanilla FuncGodot, without much trouble for being updated after new official releases.

[sinewavey]

I am open to the idea of exposing and structuring some functions in a way that makes things a bit easier to slot in, but I think C-style function pointers (the callable route) might be a bit more difficult route / out-of-the-normal GDscript expectation, though somewhat aligning with how things actually do work behind the scenes in Godot (for example, with the dummy Server classes that are almost pure abstract)

However, at the current moment, the progress is far too early to make meaningful decisions on this yet.

[RhapsodyInGeek]

Ideally consists in that every step would be split into a function for separation of concerns and easily having custom versions of it for a project.

One of the things we're trying to do is simplify Core, not add extra complexity to it. There is a lot of bloat and spaghettification in the current implementations of the build process, making it very difficult to parse exactly what is going on. The code behind the build process is extremely labyrinthine as it is.

The idea of callable interrupts or overrides in the build process is intriguing, and a spot I think could really benefit from that is adding more map formats to the parser.

That said, something like custom tool textures you keep championing, I'm still not convinced of its applications, or how any of the build processes would be able to support what you proposed, whether it be the current iteration of FuncGodot or the new Core implementation we're working on. Also, as Vera already stated, the progress is still far too early to know what opportunities would even be available to interrupt.

It also means we need to know what data we're supposed to expose to these interrupts? What steps are they supposed to occur? What does the interfacing for that look like? How do we keep the resource properties from becoming overwhelmingly cluttered? Do we add this new interfacing to the MapSettings resource or do we create yet another resource to handle the map building? Where does that new resource go so that it can be read by FuncGodot Core, which has no direct interface as it is generated purely during map build?

What are the other applications for these interrupts? I can think of some at the parsing level, to add support for more map formats, but can you suggest any other applications beyond tool textures? What are the actual specific purposes of a feature like this? We need more than just

That would allow to have custom map build solution for features that are tied to projects or personal workflows that aren't merged into vanilla FuncGodot, without much trouble for being updated after new official releases.

We need specific examples because a feature like what you're asking for is not trivial to implement. There are lots of considerations to make, and unless this feature is something that is impossible to pull off with the current systems in place, it doesn't seem to be worth the effort it'd take to implement it.

[azur-wolve]

It looks that this thread has mixed with the tool textures thread for some reason.
Just mentioned it as an example.

In the discord server have seen a Vera's screenshot with something like FuncGodotMapSlamfire.
I assume that is a custom map builder tied to the project is working on for convenience.

From that we can assume that being able to easily use FuncGodot as a library to adjust the map build step to project needs is pretty handy.
Vera knows well the codebase because is member of the FuncGodot team.
But for the rest of the user base can be a bit cumbersome, as in many occasions only want to adjust a specific aspect of the build step.
Even if a core refactoring happens, still being able to easily adjust specific bits without knowing the whole codebase or changing much code that isnt directly related to the aspect itself, is valuable.

but can you suggest any other applications beyond tool textures?

Basically any kind of customization of the map build process.
There could be users that add support for .vmf and share it as an addon that complements FuncGodot, etc.

But now that you mention it, something i've been thinking of is creating my own map editor using FuncGodot as base.
Could either use map format created from external map editors like TB to then adjust the project tied stuff within, or do it all there.
That could allow me to set the workflow i want without external restrictions like what TB supports or doesnt, etc.

From all that has been said can get that is about being able to use FuncGodot as a library.
So ideally could either use the default FuncGodotMap for generic functionality, and for more custom solutions use it as a library, via extending it either by overriding "virtual" funcs or binding custom callables, could combine both.

About how to do it... could think about something.
From first glance, splitting the aforementioned current spaghetti code into separate and reusable functions looks like the way to go.

Envisioning a set of functions that could use to create a personalized map build pipeline by using the native one as example.

In resume, being able to use FuncGodot as a library to have a custom map build pipeline.

---

About performance, as it is a concern, we have a little problem here: The initiative of keeping all code as GDScript to maximize potential contributors, expose all and make it easier to handle sounds convincing at first, but has a problem similar to what Godot has.
If making something pretty easy to use, reaches a point that the usability bottleneck will be somewhere else.
In godot for much easier you make the engine to being used, there's an inevitable curve and other struggles in gamedev that the bast majority of newcomers will face.

Transposing this to FuncGodot, the number of contributors will be limited, there's always a separation of a basic user base that only uses the built-in features, and an advanced one that is the one that looks at the source code, makes PRs, etc.

So much of the performance problem would be in the most low-level tasks, like generating the meshes and collision shapes could be done in C++ functions that are exposed on the FuncGodot GDScript API and used on the rest of the code.

[azur-wolve]

Components

• Singletons (with utility funcs, to being able to call them from elsewhere)

• Classes

  • MapBase ("abstract" base class, extend for ultimate customization)
    • FGMap (the native map node, extends MapBase, also can extend to just add on top)
  • MapPreprocessorBase
    • FGMapPreprocessor
  • MapParserBase
    • "
  • MapGeneratorBase
    • "
  • MapConverterBase
    • "

So could either use the built-in ones, extend those, or use the base ones for a fully custom solution.
The key is that would use the low-level functionality (building geometry, querying the data on the map file, etc) to do so, expected to be functions.

• Resources
  • FGToolTexture (?)
  • etc

Split reponsibilities by component

• MapParser (extracts data from map file, caches and prepares it for being used)

• MapPreprocessor (handles fgd ents, tool textures, visibility flags, custom metadata, etc)

• MapGenerator (generates meshes, collision shapes, occlusion, UVs, materials, appy textures to faces, etc)

• MapConverter (turns fgd ents into nodes / scenes, fills properties in those from TB properties)

Just take it as a rough sketch.

Side notes

• let FGDEnt resource have various callable properties to allow specific logic being called on specific steps of map build, similar to _fg_apply_properties() and _fg_build_complete()

• "FG*" for compact naming

• upcoming GDScript namespaces will prevent naming collisions

• upcoming GDScript traits will add flexibility (assumed to be "first-class citizens")

---

About how using Callables

var callable_A : Callable = default_callable # its part of the built-in pipeline, but could be bound to a custom one 
var callable_B : Callable # an optional one to add custom logic

func _specific_build_step(): # acts like a crossroad where a build step happens, can override for custom solution, or just add logic to with callable_B
    callable_A.call()
    callable_B.call() # expected to have been already assigned

• using callables would work better with dependency injection (ie passing self as first param by convention, it also allows future changes without changing the API)
• I wouldnt overcomplicate too much the built-in solution to include much callable stuff, just add one optional callable for each step, and if need a more finegrained solution, just exend the base types to create it.

[azur-wolve]

as has been mentioned, a note on tool textures

SUBORDINATION (TB side)

2 types of tool textures, regarding how they relate to fgd ents.

a) enforced: necessarily tied to a solid ent with properties in TB, so they just give visual feedback
(ie: TRIGGER → func_trigger)

b) optional: complementary to a solid ent, no properties set on TB, done from Godot instead on map build, but still can do lookup on fgd ent data, like origin_type.
(ie: ORIGIN, HULL, CLIP, SKIP)

WHAT TOOL TEXTURES DO

TB side:

• visual feedback

• transparency

• adjust properties during mapping step (to apply later during map build, ie: rotation pivot with ORIGIN)

Godot side:

• resource generation (meshes, col shapes, occlu-meshes, nav-meshes)

• node generation
  ie: TRIGGER

• set variables (collision layers, transforms, etc)
  ie: ORIGIN, CLIP_*

• rendering
  ie: SKIP, SKY

as tool textures tend to be used for the same kind of things, its easier to have an idea in which map build steps they can be involved.

here a list of potential tool textures for Godot:

- OCCLUDER (generate occluder nodes from brushes, or determine the occ mesh of a solid when the brush with the texture is inside its ent, like with ORIGIN, for finegrained occlusion culling)

- CLIP_* (NPC, BULLETS, PLAYER, GRENADES, FLYING, NAV, etc)

- NAV (navigation meshes)

- BLOCK_LIGHT (blocks all light)

- BLOCK_LIGHT_BAKE (baked light only)

- BLOCK_LIGHT_DYN (dynamic light only)

- BLOCK_FOG

- BLOCK_SHADOW

- BLOCK_PHY_PARTICLES (physical particles)

- BLOCK_VIS_PARTICLES (visual particles)

- BLOCK_SOUND (audio occluder)

- BLOCK_VIEW (blocks rendering, useful when want to )

- TRIGGER

- WATER / LIQUID

- SKY (skybox)

- HULL (generate simplified col shape for "illusionary" solids with complex visual represenation)

- NOLIT (faces dont receibe any light, either baked or dynamic)

- NOLIT_DYN (faces dont receive dynamic light)

- NOLIT_BAKE (discard faces for lightmaps when baking, can be useful when dont use CLIP or SKIP but want not use for baking, either if is about visual look or to optimize lightmap size)

- NOSHAD (faces dont receive shadows)

- NOSHAD_DYN

- NOSHAD_BAKE

- FOG (fog volume)

even more gameplay tied ones like:

For those having a way to call a custom Callable on the map build step should do the work, while still allowing specify the generic stuff like how they are treated for rendering, collision, etc.

- LADDER (shape becomes child of subordinated-to ent)

- CLIMB (similar to LADDER, like in L4D2)

- BLOCK_LOS (line of sight, for NPCs)

As can see things can complicate a bit for include some, better give a way for users to create their.
Then can share them, just like custom fgd ents are being shared.

Some can be done using fgd ents, while others not.
Others can be easily worked around, but others not.
As in many cases involves generating a node, using them like ORIGIN to let them be child of the solid ent node they're applied to.

Still pretty useful to can have a visual representation.
The problem is that TB only applies textures to a specific ent if created via custom hotkey.
That's not a big deal if have just a couple, but as the list grows..
Also final users should know that and create the hotkeys, which isn't ideal.
Having a way to just create a brush inside a solid ent with a tool texture applied to obtain a desired behavior, makes life a lot easier.

NOTE: it doesn't necessarily mean that FuncGodot should support all.
Its more about ensure that the map build steps are well defined so everyone can implement its custom solution without caring much that it breaks after future updates.
The callables for binding could help on this regard.

Example code about tool textures

To have some idea.

#FGToolTexture.gd
class_name FGToolTexture extends Resource

var call_on_buildstep_A : Callable
var call_on_buildstep_B : Callable
var call_on_buildstep_C : Callable


#FGMapSettings.gd
var tool_textures : Dictionary = { # can add more via inspector, those are FGDToolTexture resources
    "clip": preload("res://dir/to/tool_tex/clip.tres"),
    "skip": preload("res://dir/to/tool_tex/skip.tres"),
    "origin": preload("res://dir/to/tool_tex/origin.tres"),
}


#FGMap.gd
func _mapbuild_step_A():
    for tooltex in map_settings.tool_textures.values():
        tooltex.call_on_step_A.call() # that is expected to be done iterating on the faces of fgd ents on the map file, there should be a way to pass arguments.
    

ofc is not that simple
must allow to determine generation, rendering, etc.
just to have some idea.

[RhapsodyInGeek]

So much of this is way outside of func_godot's scope it's difficult to know where to begin.

I assume that is a custom map builder tied to the project is working on for convenience.

From that we can assume that being able to easily use FuncGodot as a library to adjust the map build step to project needs is pretty handy.

There's too much assuming going on here. You're better off asking @sinewavey what FuncGodotMapSlamfire actually is and does first before drawing conclusions.

Basically any kind of customization of the map build process.
There could be users that add support for .vmf and share it as an addon that complements FuncGodot, etc.

I've actually already added VMF support to the new parser, but this is so far the one place I can think of that might benefit from an interrupt or injection. That said, there's still no good suggestions for how to interface with that sort of custom tooling, and it's being set up to just as easily add new parse functions to be called within the main one. There are also not too many variants on the map file format to worry about. The only "weird" one seems to be Dark Radiant's formatting, the rest are all mostly the same data.

From first glance, splitting the aforementioned current spaghetti code into separate and reusable functions looks like the way to go.

Any real study of the code would show you that this can't actually be done with what currently exists. That's the definition of spaghetti code: it's so interwoven with each other that pulling on one piece inevitably pulls at another. That's the main impetus behind the rewrite: what can we do to simplify the core functionality.

About performance, as it is a concern, we have a little problem here: The initiative of keeping all code as GDScript to maximize potential contributors, expose all and make it easier to handle sounds convincing at first, but has a problem similar to what Godot has.

It's not going to be changed to another language. We haven't done all we can do with GDScript yet, and the benefits still outweigh the issues. GDScript may not be as performant as Native, but it's still performant. Part of our rewrite is to try to work in real optimizations. A lot of Core's processes and structs were bespoke for no real reason or benefit, a carryover from Qodot's original design in its mix of C, C++ GDNative, and GDScript.

Every single tool texture you listed not only can be done with an entity and editor tags / shaders far more easily than trying to write some sort of tool script injection, but also Godot just doesn't handle any of these things on a per face / plane basis. It's just not how you should be trying to pull off any of these mechanics listed in this engine.

A lot of what you're proposing sounds more like a completely different project to what FuncGodot's goals are intended to be, and a lot of it seems to be filled with breaking changes. None of it has obvious interfacing or integration solutions. To be clear, the Core rewrite we're doing is not intended to change the way users interface with FuncGodot: it's supposed to be invisible while fixing some longstanding bugs and hopefully improving performance by using some of the built-in Godot methods rather than the bespoke FuncGodot methods for map generation. It is not to change the nature or goals of FuncGodot or how it's used. Too many people have projects dependent upon the way things currently work.

[RisingThumb]

@azur-wolve May I ask why you can't make a new node that extends the current func godot map node, and simply override functions as you need and call super() when you need to call the original func godot map node behaviour?

Also I agree with Tim, most of these can be done as separate brush entities or as shaders. The clip_* tool textures could instead just be a bit flag property that corresponds with each collision layer you want collisions on. A bunch of these are also just GeometryInstance3D properties which there is an open issue to expand this if they are essential to your project #37

[sinewavey]

For clarity, what RisingThumb mentions is exactly what I do with an inherited class:

@tool class_name SlamfireMap extends FuncGodotMap;

@warning_ignore("UNUSED_PRIVATE_CLASS_VARIABLE")
@export_tool_button("Build Map", "Bake") var __BUILD__: Callable = verify_and_build;

@warning_ignore("UNUSED_PRIVATE_CLASS_VARIABLE")
@export_tool_button("Bake Lightmap", "Bake") var __BAKE_LM__: Callable = bake_lightmap;

@warning_ignore("UNUSED_PRIVATE_CLASS_VARIABLE")
@export_tool_button("Clear All", "MissingNode") var __CLEAR__ = remove_children; 

@export_flags("Unwrap UV:1", "Remove Persistent Nodes:2") var flags: int = 1;

@export_category("Data")
@export var persistent: Node;
@export var paintmap: Paintmap3D;
@export var particle_sdf: GPUParticlesCollisionSDF3D;
@export var nav_region: NavigationRegion3D;

func _ready() -> void:
	if Engine.is_editor_hint():
		map_settings = load("res://fgd/sf_map_settings.tres");
		add_to_group(&"navigation_mesh_source_group");
		return;
	
	if is_in_group("navigation_mesh_source_group"):
		remove_from_group("&navigation_mesh_source_group");

	process_mode = Node.PROCESS_MODE_PAUSABLE;
	return;

func _add_to_scene(node: Node, new_parent: Node = self) -> void:
	if !node || !new_parent: 
		return;	

	if !node.is_inside_tree():
		new_parent.add_child(node);

	var scene_root := get_tree().get_edited_scene_root();
	node.set_owner(scene_root);
	return;

func _build_complete():
	reset_build_context()
	stop_profile("build_map")
	print("Finishing SLAMFIRE build...");
	
	if !persistent:
		persistent = Node.new();
		_add_to_scene(persistent);
		persistent.set_name("PersistentNodes");
		
	if (flags & 1):
		unwrap_uv2();

	# post build creation like ParticleSDF and NavigationMesh 
        # depending on entities like navigation obstacles that will be removed
        <removed for brevity>
	
	for child in get_children():
		if child_is_temporary(child):
			remove_child(child);
			child.queue_free();
			continue;
		child.set_display_folded(true);
		child.emit_signal(&"script_changed") # hack to actually update the display folding
	
	print("Build complete!\n")
	emit_signal(&"build_complete");
	return;

func build_entity_nodes() -> Array:
	if !paintmap:
		<removed for brevity - set up and bake paintmapping>
	if !particle_sdf:
		<removed for brevity, same thing with particle SDF collision>
	return super.build_entity_nodes();
 
func apply_properties_and_finish() -> void:
	super();	
	for entity_idx in range(0, entity_nodes.size()):
		var entity_node: Node = entity_nodes[entity_idx] as Node
		< removed safety / null checks for brevity>
                # generic properties that pretty much any entity can take
                # can be handled here instead of in each class
		var props: Dictionary = edict["properties"];
		for property in props:
			var value: Variant = props.get(property);
			match property:
				"targetname":
					var targetname: String = props.get("targetname", "")
					if !targetname.is_empty():
						entity_node.add_to_group(targetname, true)
				"detail_text":	
					FFWD.set_detail_text(entity_node, str(value));
	return;

func remove_children() -> void:
	for child in get_children():	
		if child == persistent:	
			if !(flags & 2):
				continue;	
			persistent = null;
		elif child == paintmap:
			paintmap = null;
		remove_child(child);
		child.queue_free();
	return;

func bake_lightmap() -> void:
	if paintmap:
	        paintmap.prepare();
	return;

func child_is_temporary(n: Node) -> bool:
	if n is NavObstacle:
		return true;
	return false;

[azur-wolve]

then its about of extending the native types