# Unreal Engine 4.20 & 4.21 and beyond



## malavon (Jul 20, 2018)

It's that time of the year again, with a new Unreal Engine 4 released by Epic and I'd like to take the opportunity to inform you about some
of the changes.

*What has changed*
I've been busy taking on a a huge amount of loose ends, although that hasn't really translated in much more commits due to the way I work.
Most of these I won't talk about because they're rather boring, but here are some highlights:

*3rd party builds*
Context: I'm not talking about 3rd party libraries here, I'm talking about building UE4 on your personal computer or build farm!

You read that right, I've been putting a lot of time in removing many manual steps I still had. It means you no longer have to trust me and
can build your own binaries (after reviewing all code changes of course ).
See the next post in this thread for more information.

*Single-channel sound issue*
Context: In the previous version sound was played on a single channel when using stereo speakers/headphones.

Edit: it seems this has been fixed with 4.21.2.
This has been since identified as both an issue with the code and possibly an issue or incompatibility with FreeBSD's channel mixing.
In short (more details, see sources or ask me):
Unreal Engine tries to set up 6-channel audio, which is converted into 4-channel audio by SDL's OSS code.
These 4 channels are then mixed (presumably by FreeBSD) where both the left and right channel get mapped to the left stereo channel,
with the right stereo channel getting nothing but silence.

Solution (as root, with # replaced by your pcm number):
`> sysctl dev.pcm.#.bitperfect=1`
Setting this sysctl to 1 means "The pure sound stream will be fed directly to the hardware." without any conversion/matrixing.
Note: I suspect that playing 5.1 content on stereo headphones with this setting like this will result in sub-optimal (or garbled) audio.
Set back to 0 after playing.

*Broadcast messaging*
Context: Broadcasting works differently on FreeBSD compared to Linux, Mac and Windows. This meant that it was not possible to do LAN broadcasting.

Since previous version I have modified the code in such a way that this now works on FreeBSD as well. The only catch is that only the first interface with
broadcasting enabled will be used, however this should be ok for most people.
Check your ifconfig to see if broadcast is enabled in case you need to troubleshoot this:
`> ifconfig
em0: flags=8843<UP,[B]BROADCAST,[/B]RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
...`

*Executable size*
This isn't on my side, but it's important enough to mention it.
Epic has made some changes to the build system for Linux and I've done the same on the FreeBSD port.
In short: executables are now stripped, with debugging symbols in a separate file. Gone are the days of 1Gb+ executables (although they're still rather big).
I have tested that this means I can remove the debugging symbols to make the downloads slightly smaller.

*Vulkan support*
Technically there is no reason vulkan shouldn't work right now, although I haven't yet tested it (assuming I can get it to work on FreeBSD 11.2).
If you have it running already, try adding the "-vulkan" flag (without quotes) when starting the applications. It would be nice to know if it actually works.

*Marketing *
Yes, you read that right. While it completely doesn't matter, the previous version didn't really advertise it was running on FreeBSD.
This has now been changed, although it won't be visible in the precompiled demos I supply since the logging is disabled for performance reasons.

*A ton of bugs and small improvements*
Not much comment here, too many to remember anyway.

*Demo & games*
Demos have been compiled on FreeBSD *11.2*-RELEASE, *amd64* architecture. No guarantees that they'll run on 11.1 or 11.0.
EDIT: They were also tested on FreeBSD 12.0-RELEASE-p2, with the compat11x-amd64 package installed and ran well.
I have removed all debug symbols from the binaries to reduce size and most likely you won't need those. Should you experience crashes or something,
you can download the necessary files by appending "-debug" to the filename (before the extension).

InfiltratorDemo
Rather heavy tech demo to showcase some rendering features of the engine. Always nice to use as a little gpu roaster and on cold winter nights.
The official Epic documentation has more information on how it came to be.
There's also a FreeBSD 12 version, compiled with Unreal Engine 4.21.2 now.

PlatformerGame
More information can be found in the official Epic documentation.

ShooterGame
Thanks to the multicast code changes mentioned above, this game is now *multiplayer* in LAN as intended.  Up to 8 players and bots can fight to the death!
The official documentation has more information about this sample.

StrategyGame
Rather fun little tower defense game, more information in the official documentation.

VehicleGame
Offroad vehicle game, beat your own time record over and over again! Or not 



Spoiler: SHA512 hashes



SHA512 (UE4.20-FreeBSD11.2-InfiltratorDemo-debug.tar.xz) = a59270d2d1bf12ea0565187cdbd94277bb5c3db58f9984c442b4200b9dcad6e7da4b16399db6b4496e7d8d4832e439527fd20fa961021336603a31ad7a3efcf3
SHA512 (UE4.20-FreeBSD11.2-InfiltratorDemo.tar.xz) = aad49c0d1d150d4bbaffaa5cd03c5f2865b50639ce19590f52ef68d118c54a9d3efa2a6fd1f581728f99a0201ced7c404bf343829c10d1df057c7a3b358974cf
SHA512 (UE4.20-FreeBSD11.2-PlatformerGame-debug.tar.xz) = da60bf5edd8f92cf9e739c191cebac2532ab10da388eb55afb0eb50e47a91c500bea9d6a74d4a8dd4d4306f50bb94615e9dbabc0e035d978142f968d9b8476f7
SHA512 (UE4.20-FreeBSD11.2-PlatformerGame.tar.xz) = 9e79b51532fd04244debff7518b3bd84487c22999446feb1ffd88aff30d5985bc98785de25c46254e39dc7e30491cbb266bc4974decb2eb30a7b7b7b2d895e56
SHA512 (UE4.20-FreeBSD11.2-ShooterGame-debug.tar.xz) = 8a9b6b24355a6f23058af4da105bf8b323d7799438650c515d5610abdd973036b0a16ae6cdd6c69eda4f7d614ff76d0705a1fa3348acc57d546093c55a417d3e
SHA512 (UE4.20-FreeBSD11.2-ShooterGame.tar.xz) = c42dd16078c7a1fc3fbb218c28ff1312795dbea2fa02e0dd9e167323d7f6fcd27de3ee611905f50f71964cc04a76f0914883d10b95938bcb2e30de111b4314ba
SHA512 (UE4.20-FreeBSD11.2-StrategyGame-debug.tar.xz) = 16f78e6eedd4e55fd64f40e5248517aaf2499f8ab4f3681ac42b41f0df44f21d03694a27809af4c5e3bfba6e671af97cb88e917161a236464d61c4674d186cfe
SHA512 (UE4.20-FreeBSD11.2-StrategyGame.tar.xz) = 189185bf09c91a8f3134efd7b07541466fb571a63d6d7c6c03bf7b9ab45ab0837062a8b5c88711a16907801be57d68028cc704bd07a45e23dfc35eefb7d3ff79
SHA512 (UE4.20-FreeBSD11.2-VehicleGame-debug.tar.xz) = 47d1fb37d9be80b87428741b3011d0e7e2c65fb6a5f927cdce4c63db39196d64bf12b87d9997e64e86b5a284535144fbbbb9e7f8d7c732cab6a9a2dd4056329c
SHA512 (UE4.20-FreeBSD11.2-VehicleGame.tar.xz) = 4d6692486884cd5e2468e6b9a0d92cba3aa36dfd2386901fc1014909d468c5f2dbf0eb7bc5c6876ed9a29fdd27a375c5227df0b3dea14852c726db4cb632aa47



*Practical stuff*
In case you need to change keymappings (I don't use Qwerty but tried setting them back to original), change in:
<name>/Config/DefaultInput.ini e.g. ShooterGame/Config/DefaultInput.ini

* You will probably need an OpenGL 4.3-capable GPU and driver, although I've cooked for OpenGL 3 and vulkan as well
* If a demo starts on only part of your screen, try alt+enter twice (once to get out of fullscreen mode, second time to go back in)

Code on GitHub


----------



## malavon (Jul 20, 2018)

How to build Unreal Engine 4 on FreeBSD
Note: I have hardcoded /usr/local as prefix in several places. If you're using a different $PREFIX, you'll have to modify the code.

*Prerequisites*
Both running and building require at least 16Gb of ram (with 20Gb of swap space preferrably on an SSD). More ram is always better.
Do not skip any steps or you will have a hard time debugging why you're not getting anywhere.
OS version: FreeBSD *11* or *12* for *amd64*, tested on 11.2 and 12.0
I'm pretty sure that there shouldn't be any problems doing all this on 11.1.
EDIT: added 12.0 as "supported"

Get set up using the official UE4 GitHub documentation. Without doing this, you won't be able to see the repository.

Install the necessary packages to build the engine and its dependencies.
You might be able to get away without installing git and working with the GitHub zip, although I haven't tested this. Otherwise install it.
`> pkg install git`
Install the necessary packages to compile and run the engine tools.
`> pkg install autoconf automake bash cmake coreutils gmake mono libinotify libtool unix2dos xdg-user-dirs xdg-utils xorg`
From this point on you don't need root privileges anymore, and you have to relinquish them or the tools won't work!

*Clone & first setup*
Note: if your shell doesn't support pushd/popd you can use cd and just return to the previous directory on each popd.
Start by cloning the repository (unless you're working with the zip).
`> git clone git@github.com:malavon/UnrealEngine.git`
Do the setup, this will download third party libraries and engine content.
Note: check this forum thread to see if there are any newer versions available than the one in the git command.
`> cd UnrealEngine
> git checkout 4.21.2-freebsd
> ./Setup.sh`

*Build all thirdparty libraries*.
The FbxSdkStub is only needed from 4.21 on. In the 4.20 tags/branches, there's a binary compiled version which I removed once I got to 4.21.
`> pushd Engine/Build/BatchFiles/Linux/
> ./BuildThirdParty.sh --all
> ./BuildThirdParty.sh -b FbxSdkStub
> popd`
If these steps don't end in "Success", check BuildThirdParty.log to see what went wrong. Building separate libs can be done using
`> ./BuildThirdParty.sh -b <name>`
You may encounter errors if you try rebuilding all dependencies a second time, it's advisable to rerun Setup.sh before doing this.

*One manual step*
The engine doesn't take care of building this binary itself, which is probably an oversight on Epic side after adding this in UE4.20.
`> pushd Engine/Source/Programs/BreakpadSymbolEncoder
> ./BuildBreakpadSymbolEncoderFreeBSD.sh
> mv BreakpadSymbolEncoder ../../../Binaries/FreeBSD
> popd`

*Generate makefile*
Feel free to omit the -cmakefile. You can also omit all arguments, which means the generator will create all make & editor files.
Supported are e.g. cmake, codelite, kdev4 (kdevelop) and some others. Check the documentation or code for these flags.
`>./GenerateProjectFiles.sh -makefile -cmakefile`

*Build the editor and tools*
`> make`
Yes, that's all :-D
This command has built the most important tools, using the "development" build. Check the makefile for other things you might build or for debug/shipping builds.

*Start the editor the first time*
The editor will spawn several ShaderCompileWorker processes and use them to compile the default materials/shaders in the engine.
The same will happen when you use the editor to load an existing project for the first time.
`> pushd Engine/Binaries/FreeBSD
> ./UE4Editor`

Enjoy!

Notes:
* if you want to use a branch (e.g. 4.21) instead of the tag, you'll notice that I regularly rebase my repo. In that case you'll have to do a
`> git fetch origin && git reset --hard origin/4.21`
* if you update your branch or checkout a different one (e.g. master), you'll have to recompile your tools and shaders.


----------



## ShelLuser (Jul 20, 2018)

Why not consider to adopt irc/unreal? It's unmaintained right now and I get the feeling that you have a bit of an affinity with it and FreeBSD which could make you a perfect candidate to keep it updated.

Maybe food for thought?


----------



## shkhln (Jul 20, 2018)

malavon said:


> there is no reason vulkan shouldn't work right now, although I haven't yet tested it (assuming I can get it to work on FreeBSD 11.2).



Relevant bits: ICD loader and Mesa patches. Nvidia doesn't provide a native ICD for FreeBSD at the moment and it's not clear whether they are planning to do it; there is a possible workaround (more on that later).


----------



## kpedersen (Jul 20, 2018)

ShelLuser said:


> Why not consider to adopt irc/unreal? It's unmaintained right now and I get the feeling that you have a bit of an affinity with it and FreeBSD which could make you a perfect candidate to keep it updated.



I don't think that port is what you think it is. It has nothing to do with 3D engines. It is actually an (The most popular?) IRC server 

I did get excited though for at least 3 seconds whilst the freshports page was loading. So thank you for that


----------



## malavon (Jul 20, 2018)

shkhln said:


> Relevant bits: ICD loader and Mesa patches. Nvidia doesn't provide a native ICD for FreeBSD at the moment and it's not clear whether they are planning to do it; there is a possible workaround (more on that later).


Yeah, I've been looking into it and I get the feeling that my (Haswell) gpu isn't really supported. I'll do some more digging, but if that's the case I won't be able to test vulkan; I'm not counting on Nvidia for releasing a vulkan-ready driver soon (although I would definitely appreciate it).



kpedersen said:


> I don't think that port is what you think it is. It has nothing to do with 3D engines. It is actually an (The most popular?) IRC server


I was wondering the same and more or less waiting for an error to be rectified. Or if not was trying to understand it humoristically/sarcastically somehow.


----------



## shkhln (Aug 5, 2018)

shkhln said:


> Nvidia doesn't provide a native ICD for FreeBSD at the moment ... there is a possible workaround (more on that later).



So, about the promised workaround... Have you noticed FreeBSD's dynamic linker makes no effort to differentiate native FreeBSD shared libraries from Linux (glibc) shared libraries? It  merely complains about missing symbols on accidental mixup. More so, libmap.conf man page actually contains this curious usage example (a reference to the defunct linuxpluginwrapper port):

```
# Glue for Linux-only EPSON printer .so to be loaded into cups, etc.
     [/usr/local/lib/pips/libsc80c.so]
     libc.so.6               pluginwrapper/pips.so
     libdl.so.2              pluginwrapper/pips.so
```

I cleaned up and dumped on GitHub my crappy attempt at a similar compat library targeting specifically Nvidia's Linux Vulkan ICD: https://github.com/shkhln/nvshim.


----------



## kpedersen (Aug 5, 2018)

shkhln said:


> a similar compat library targeting specifically Nvidia's Linux Vulkan ICD



Thats actually a really cool idea


----------



## NuLL3rr0r (Aug 6, 2018)

@*malavon* thank you so much for the effort. UE4 game dev here.  The only reason I moved away from FreeBSD on desktop was the lack of official support for UE4 on FreeBSD (I'm running Funtoo Linux on desktop at the moment). If I recall correctly there was another GitHub repo up to UE 4.16. But, it looks like you made a great progress. I should definitely find some time and check this out.

Good old FreeBSD + i3wm (+ UE4) would be an exciting development machine. Thank you and keep up the good work!


----------



## malavon (Aug 6, 2018)

shkhln said:


> I cleaned up and dumped on GitHub my crappy attempt at a similar compat library targeting specifically Nvidia's Linux Vulkan ICD: https://github.com/shkhln/nvshim.



I'm going to play around with this somewhere in the following days. It's really a great idea and if it allows using vulkan on FreeBSD with Nvidia cards it'll help me get that part of UE4 right as well.
Many many thanks shkhln for creating this!


----------



## malavon (Aug 6, 2018)

NuLL3rr0r said:


> UE4 game dev here. The only reason I moved away from FreeBSD on desktop was the lack of official support for UE4 on FreeBSD (I'm running Funtoo Linux on desktop at the moment). If I recall correctly there was another GitHub repo up to UE 4.16. But, it looks like you made a great progress. I should definitely find some time and check this out.


Great to see there's a game dev in the forum. Wish I could say I was one myself, but who knows what the future holds. 
If you test it out, it'd be great if you could report what actually works and what doesn't. Some features of the editor won't work (FBX import for one) and there are bound to be many like that since I don't test them. 
Feel free to create issues in the GitHub, with guidelines on how to reproduce the problem and I'll take a look at it when I can spare the time.


----------



## malavon (Aug 11, 2018)

shkhln I've been able to use your shim to run with vulkan support! There seem to be some graphical issues (mainly with the UI),
not sure if they're caused by UE4 or the Nvidia driver.
Nevertheless I know now that it works and I'll try to find out if the issues are also present on other systems.

You might actually want to start another thread about this specifically, I'm sure it'll be interesting to more people than the ones interested in UE4.

Interesting note: framerate under vulkan more than doubles. I'm not sure if it's inherent to vulkan or that there are simply certain
graphical effects that are disabled and thus improving the framerate.


----------



## shkhln (Aug 11, 2018)

malavon said:


> shkhln I've been able to use your shim to run with vulkan support!



Glad to hear that! I'm now reviewing binary compatibility for structs I'm currently passing to FreeBSD's libc without conversion and I'm myself a bit surprised it actually works  I'll fix it though.



malavon said:


> Interesting note: framerate under vulkan more than doubles. I'm not sure if it's inherent to vulkan or that there are simply certain
> graphical effects that are disabled and thus improving the framerate.



Twice the OpenGL framerate is clearly too much. Something must be botched with regard to that measurement.


----------



## shkhln (Aug 11, 2018)

malavon said:


> There seem to be some graphical issues (mainly with the UI),
> not sure if they're caused by UE4 or the Nvidia driver.



Vulkan is still under heavy development, you should try the 396.x branch. (Don't download the driver directly, update the version number in the nvidia-driver port and recompute the checksum with `make makesum`.)


----------



## malavon (Aug 13, 2018)

shkhln said:


> Vulkan is still under heavy development, you should try the 396.x branch. (Don't download the driver directly, update the version number in the nvidia-driver port and recompute the checksum with  make makesum.)


Tried these, no changes. That makes me think that it might still be a UE4 issue. 
Also, it seems that vulkan uses the mobile renderer instead of the desktop one, supporting less features. This might explain why the framerate is so different.

Found a future task in the UE4 roadmap, it'll be interesting to see when this actually happens. I think it's been there quite some time already.


----------



## malavon (Aug 24, 2018)

Just a quick update. I try to keep up with releases made by epic, and that's also true for 4.20.1 and 4.20.2. 
Both have freebsd ports as well, with 4.20.2 released (for FreeBSD) today.


----------



## malavon (Oct 2, 2018)

4.20.3 has been released a while ago and I updated the github with a FreeBSD version.


----------



## malavon (Jan 5, 2019)

It took me a while, but I managed to whip up 4.21.0 and 4.21.1 as well.
In this release, epic has changed the default for Linux to vulkan as opposed to opengl and it caused me some headaches. It did encourage me to do some of the clean up I've been wanting to do though.
Since the build & run process on FreeBSD hasn't changed, I didn't want to be creating another topic for it on this forum. Instead, I chose to update the post to guide compilation.

The caveat is that the editor now needs to be started with -opengl flag, otherwise it'll choose to start with vulkan.Until vulkan support is officially in the nvidia driver, this won't work for most people.
Edit: I think I changed the default configuration correctly and it'll still start using OpenGL on FreeBSD. If it does crash inside a Vulkan class, just provide the option 
For people who would like to use vulkan (which is not fully implemented in UE4 yet!), check out earlier posts in this forum thread for a really neat solution.

One thing of note is that the very last binary is now gone. It's now required to build the FBX stub library before building the editor to prevent linker errors.


----------



## kpedersen (Jan 5, 2019)

Very cool. Looking forward to giving it a whirl!

Btw, any plans for a port at some point? They are a bit of a faff to make initially but is the build process for UE4 on FreeBSD starting to calm down and stabalise? It might be worth it.


----------



## malavon (Jan 7, 2019)

kpedersen said:


> Btw, any plans for a port at some point? They are a bit of a faff to make initially but is the build process for UE4 on FreeBSD starting to calm down and stabilise? It might be worth it.


Not really. Originally I intended to do so, but that was before I knew the complexity of UE4 and its build system.
There are a few hurdles I can think of:
* RPATH/RUNPATH issues inside the shared libs and executables. Most of the thirdparty dependencies - I just fixed one a few days ago - have absolute paths to find their dependent libraries. That would break any port for sure.
* Possibly the engine needs to write inside it's own directory while building a UE4 project. I'm not 100% sure though, I noticed this a long time ago while I was trying things out. Might have been an issue in my port back then.
* Also, I'm a bit short on time I can invest in this. I'm afraid that a port will make it worse without much returns.

But the really big issue I think is the license. The best reason I can think of to create a port is to be able to install using `pkg` and that's simply not possible with the current license anyway.

I cant see it happen in the near future, but never say never


----------



## kpedersen (Jan 7, 2019)

malavon said:


> * RPATH/RUNPATH issues inside the shared libs and executables. Most of the thirdparty dependencies - I just fixed one a few days ago - have absolute paths to find their dependent libraries.


I'm not entirely fluent at the porters handbook but surely hardcoded paths to stuff in /usr/local/* is acceptable. I don't think the ports system is portable to different prefixes in may ways.



malavon said:


> But the really big issue I think is the license.


Ah of course, I completely forgot about that "small" detail haha.

Yeah I understand your reasons. I think at most a binary could be hosted on Epic's servers (or release build on GitHub) that the port just wraps like a few other proprietary and Linuxulator ports.


----------



## malavon (Jan 7, 2019)

kpedersen said:


> I'm not entirely fluent at the porters handbook but surely hardcoded paths to stuff in /usr/local/* is acceptable. I don't think the ports system is portable to different prefixes in may ways.


It's worse. The paths are to the build directory itself. It would be /usr/ports/xx/ue4/work/... or something, which wouldn't be very portable.
Not that it can't be fixed for at least the most parts, but I'm not focused on it so it's a slow process.

As far as the license goes, it's what kept me from porting Unreal Tournament as well. That and the fact that UT has been more or less abandoned by Epic with no community members allowed to take over.
With the help of someone on the UT forum I got pretty far. It ran standalone and I could actually play. 
Then during staging I saw missing files and during my investigations I noticed not everything was open-sourced. On top of that the license also forbade me from distributing the FreeBSD binaries as well.
Both of these killed all enjoyment I had for the (UT) project.
We could have been talking about FreeBSD UT tournaments right now :-D What a missed chance!


----------



## kpedersen (Jan 7, 2019)

Ah what a shame! Guess we will have to stick with Doom tournaments for another 50 years then haha.

I have seen some pretty funny ports in my time (was it Intel C++ compiler? perhaps not even FreeBSD ports but maybe NetBSD pkgsrc, cannot recall) which actually requested a license file as it was building, could be quite funny to get the port to open up a web browser and get the user to jump through all of Epic's hoops before building UT for them .

But yeah, certainly not a good use of your time. Lets wait and see if Epic gains more interest in FreeBSD one day.


----------



## malavon (Jan 14, 2019)

I have added a bit of extra OS information in the original posts since I'm currently running 12.0-RELEASE(-p2)
Everything seems to be fine running on 12.0 with compat11x, but building is currently not possible due to an incompatibility of the OpenSSL versions.
I used to refer to the system's (base) OpenSSL version, but it seems OpenSSL 1.1 and 1.0.2h aren't fully compatible.
I'll update this on the 4.21 branch, but the "4.21.0-freebsd" and "4.21.1-freebsd" tags won't build on FreeBSD 12.0.
4.21.2 will be fine (which is due any day now anyway).

There are some more improvements to the FreeBSD port to come as well, more on that later.


----------



## malavon (Jan 24, 2019)

*Unreal Engine 4.21.2*
And ... there's another release of the engine. 4.21.2 was released 2 days ago and the FreeBSD port is also ready. This time I'm right on the ball 

There are quite a few improvements in the port. Anyone wanting to test out UE4 should definitely use this one above all others.

*Single sound channel issue*
It seems Epic has fixed this issue upstream and now you can have glorious 2/5/7 channel sound without setting any sysctls.
I only have stereo hardware though, so I can't test surround sound.
The Infiltrator demo has been recompiled for FreeBSD 12 (see first post in this thread) and I'm hoping people can test this out on various hardware.

*Voice*
Another addition in the port is that now the Voice module compiles on FreeBSD as well.

*Version selector*
While I don't use it myself, this is a small application that's used to switch between UE4 installations on Linux. It has now been fully ported
to FreeBSD as well and allows people to stay closer to the Linux guides and have an overall better experience.

*Regressions fixed*
When I released 4.21.0 and 4.21.1 several issues were (re)introduced with upstream code changes and merge issues. I managed to identify them
and (re)apply fixes where applicable.

*Difference between Linux and FreeBSD *
I'm doing my best to keep FreeBSD in line with Linux, which wasn't easy before and has been the cause of earlier mentioned regressions. There are now
a few shell scripts in the repository to make this more consistent and improve the overall quality.

*Known issues*
As far as I can see, I believe that users using IPv6 and IPv4 mapping (which is disabled by default), might have certain networking issues.
I can't test this myself since I don't use IPv6 in my network, but I'll try to verify this somewhere in the future.

I did notice that vulkan has become quite a lot slower than it used to be, with fps dipping below opengl. This is most likely due to having
more features in the vulkan renderer than before.


----------



## Beastie7 (Jan 24, 2019)

This is really exciting work. Thanks so much for this. As someone who plans on starting game development on FreeBSD - this is great to hear.

I have question. Can this engine be used for simple 2D games with just a text editor? Also, have you came across anything that's missing in FreeBSD that could make it a better game dev platform? ie Sound servers, graphics libraries etc, etc,

Thanks.


----------



## malavon (Jan 25, 2019)

Beastie7 said:


> Can this engine be used for simple 2D games with just a text editor?


No, not really. Whatever you'd develop in UE4 you'd still need to import your assets, sounds etc using the editor. By my knowledge there are no command line tools for this.
As far as 2D goes, there is a module named Paper2D which apparently is pretty powerful but it's not being developed any longer. It's not deprecated or so, it's put in the hands of the community.
If I were to make a 2D game I'd go fake 3D (fixed orthogonal camera on the side/top) instead of actual 2D. It's more powerful, but takes more resources.
That said, for pure 2D games there might be better options out there aside than UE4.



Beastie7 said:


> Also, have you came across anything that's missing in FreeBSD that could make it a better game dev platform? ie Sound servers, graphics libraries etc, etc,


Yes and no. There are quite a few features missing because I don't have compiled the required libraries on FreeBSD yet (most notably and annoying CEF3).
But as far as FreeBSD ports go, there isn't much missing. UE4 is mostly statically linked to libraries compiled (and sometimes modified) specifically for the engine. It's more a question of stuff compiling on FreeBSD
than actually having stuff in FreeBSD, if that makes sense.

From Epic's perspective the FreeBSD port is missing cross-compiling support - mainly because I don't see the value of it. I'm in the 'set up the OS you need in a VM' camp.


----------



## kpedersen (Jan 25, 2019)

malavon said:


> I'm in the 'set up the OS you need in a VM' camp.



This actually follows on quite well from Beastie7's question. UE4 (and Unity) with their editors makes cross compiling in a VM a pain in the butt (in the way that they both require more functionality than a virtualized GPU can provide). Both have "headless / dummy" display modes but sometimes you need to tweak a few settings which is very awkward if the editor is required to do so.

So I guess I am from the "use a ratty computer set up in the corner" camp 

Which is why your great work on porting UE4 to FreeBSD is an absolute godsend for me. I can run my labs and lectures on the OS I actually like because VMs were not at all an option!

As a side effect, you have single handedly allowed me to justify removing all traces of the toy known as Unity from my unit


----------



## malavon (Jan 25, 2019)

kpedersen said:


> UE4 (and Unity) with their editors makes cross compiling in a VM a pain in the butt (in the way that they both require more functionality than a virtualized GPU can provide)


I'd personally use the VMs as build server, without any editor at all. So when I'd have to make a change, I'd do it on my local machine, commit/push and then checkout/pull it on the build VM(s). 
Certainly, if you'd want to run the editor I wouldn't recommend doing it in a VM. For building (compiling, cooking, staging, deploying) headless - as far as I know - the best tool is UE4's AutomationTool, with a command like this:
`mono ~/development/UnrealEngine/Engine/Binaries/DotNET/AutomationTool.exe -ScriptsForProject=~/development/unrealprojects/InfiltratorDemo/InfiltratorDemo.uproject BuildCookRun -project=~/development/unrealprojects/InfiltratorDemo/InfiltratorDemo.uproject -noP4 -clientconfig=Shipping -serverconfig=Shipping -utf8output -platform=FreeBSD -targetplatform=FreeBSD -build -cook -map= -unversionedcookedcontent -compressed -stage -deploy -cmdline= -Messaging -device=FreeBSDNoEditor@limbo -addcmdline=-SessionId=001812BD08D681FB6A61747CE8172C4D -SessionOwner='benny' -SessionName='limbo'`
Less resources are consumed this way, because the editor is a bit of a resource hog.
You can get this command line if you run 'UnrealFrontend', let it run for a few seconds/a minute and then copy the command line (with the copy button). Don't close the window till you've pasted it though.


kpedersen said:


> Which is why your great work on porting UE4 to FreeBSD is an absolute godsend for me. I can run my labs and lectures on the OS I actually like because VMs were not at all an option!


Wish I was still in uni, so much more interesting stuff to learn than when I was young.
It's great knowing there's someone out there using this, makes it easier to dedicate time to this port.


----------



## malavon (Apr 14, 2019)

A few days ago I ported 4.22.0. As always, Epic is frantically fixing bugs and 4.22.1 will probably surface in less than a month.
Not much more to tell at the moment. Tests passed just like they did on 4.21.x, projects still compile and run


----------



## kpedersen (Apr 15, 2019)

Nice work 

I know there was a big push to get compile times faster by integrating Live++ into the pipeline. Is this just on the Windows build do you know?

Faster compile times (for projects) would be a massive bonus for me; gets a bit awkward during lectures having to wait up to a minute for a compile to demonstrate a small code change. I often have to prepare a slide of "filler" but to keep their minds from wondering XD.

.cpp files aren't so bad but headers with the additional ubt / uht processing step can be quite slow.

Tim Sweeny was on reddit looking at interest in a scripting language due to the compile times (and because Blueprint is a bit naff). Obviously all the language buffs kept chiming in with sodding Python but in reality I would very much like to have seen something like Cling (CERN's C++ scripting language based on LLVM) being used. That way faster iteration times and yet release builds can compile into 100% machine code rather than any interpreted parts all whilst keeping with a single homogeneous language (and kill Blueprint / other Kismet replacements ).


----------



## aht0 (Apr 17, 2019)

Thanks for the awesome work!


----------



## malavon (Apr 17, 2019)

I've been looking into what kpedersen mentioned, took me a while to get back here to report on it but I wanted to be able to give actual information .



kpedersen said:


> I know there was a big push to get compile times faster by integrating Live++ into the pipeline. Is this just on the Windows build do you know?
> 
> Faster compile times (for projects) would be a massive bonus for me; gets a bit awkward during lectures having to wait up to a minute for a compile to demonstrate a small code change. I often have to prepare a slide of "filler" but to keep their minds from wondering XD.
> 
> .cpp files aren't so bad but headers with the additional ubt / uht processing step can be quite slow.


To be fair I had seen stuff about compile time improvements during development, but assumed Live++ was Windows only. I now looked into this and I found that Live coding is compiled only on the
Windows version of the editor. So at least right now this isn't supported on any of  the other platforms (Mac, Linux and FreeBSD). The code can be found under `Engine/Source/Developer/Windows/LiveCoding`
which is a dead giveaway.
I am wondering if it would be a lot of trouble to monitor the filesystem and auto-compile when something is changed. Then again, the editor really doesn't like it when something that's compiled
runs into an assert somewhere or any other type of exception. So I'm not sure this would actually be a good thing.

I did get the feeling during development that some things were faster, but I'm not sure about the entire compile process actually being faster as mentioned on the 4.22 release notes. If someone is
able to verify this by compiling 4.21.2 first and then compiling 4.22.0, please let us know in this thread.
Some things like header processing are definitely faster (or at least feel faster), but I didn't do any measurements so can't quantify this.



kpedersen said:


> Tim Sweeny was on reddit looking at interest in a scripting language due to the compile times (and because Blueprint is a bit naff). Obviously all the language buffs kept chiming in with sodding Python but in reality I would very much like to have seen something like Cling (CERN's C++ scripting language based on LLVM) being used. That way faster iteration times and yet release builds can compile into 100% machine code rather than any interpreted parts all whilst keeping with a single homogeneous language (and kill Blueprint / other Kismet replacements ).


That would explain all the "NODE_JS NOT FOUND" and "PYTHON NOT FOUND" error messages that the build system has been giving since a few months. Since everything kept working I didn't pay much
attention to these yet. But if this is because of new scripting languages, I'm pretty sure there will be a demo soon to demonstrate some of them. I'll fix this then, but it's good to know it's coming.

Note about blueprints: I'm not exactly a big fan of it but looking over the UE4 forum it would seem a lot of people do use them, especially artists with little to no coding experience.
While scripting languages could be a solution to some people, I'm pretty sure it wouldn't be for most of them (especially not something that looks and behaves like C++).


----------



## kpedersen (Apr 19, 2019)

Many thanks for looking into that. It is really fantastic that you are able to keep your FreeBSD porting work in sync with upstream.

Oh well, I can deal with the compile times, I just need to be a bit smart about it.
For my own uses, I also need to shake the habit of *:!make* every couple of lines 

No problem with Blueprints; so long as I don't need to use it. I still get a bit shakey using the Material Editor. I wish they would use a transpiler like NVIDIA's Cg rather than force me to use a node based system that will change each year but oh well.


----------



## malavon (Apr 28, 2019)

Told you it wouldn't take long  4.22.1 was released a few days ago and I just finished porting.
I am however not making a tag just yet. I had some troubles with the .NET binaries in the project which didn't get rebuilt as I expected they would have been. 
Result is that I'm not getting an AutomationTool.exe which I I use in my tests.
There is a branch 4.22.1 if someone would like to give it a try and see if it's only on my setup.

I did some compile time comparisons with 4.21.2 and found the following (first line is output from build system, second output from `time`):
*4.21.2*
Total build time: 2625,20 seconds (Local executor: 2552,72 seconds)
20111.056u 884.300s 52:24.31 667.7%     47950+3410k 1963241+492857io 553175pf+35w
*4.22.1*
Total execution time: 3194,43 seconds
23442.020u 1010.971s 1:03:46.96 638.9%  47816+3396k 2212565+564964io 645849pf+57w

Procedure used:
1. `make ARGS=-clean`
2. `./Setup.sh` (to clean all third party libraries)
3. rebuild all third party libs (see second post in this thread)
4. `time make`

Not scientifical at all. Take it for what it's worth. I wouldn't exactly call it faster


----------



## malavon (May 7, 2019)

I was lured back here by that thanks and remembered that I hadn't said what the issue was.
The same as always, I just forgot to run `UnrealFrontend` to create the AutomationTool. 

So the tag has been on the Github repo for 10 days now.


----------



## malavon (May 24, 2019)

And yet another release from Epic, 4.22.2 is now a fact on GitHub.
Make sure to recompile third party dependencies (or at least Breakpad) or you'll notice segmentation faults.


----------



## mgp (Jun 21, 2019)

malavon, your https://github.com/malavon/UnrealEngine/ repo doesn't seem to exist? What happened?


----------



## shkhln (Jun 21, 2019)

mgp said:


> https://github.com/malavon/UnrealEngine/ repo doesn't seem to exist?



It's a fork of a private repo. See https://www.unrealengine.com/en-US/ue4-on-github.


----------



## malavon (Jun 21, 2019)

Make sure to use either a release tag or the 4.22 branch (or 4.21 etc). The master branch doesn't compile currently.
Epic has a habit of fixing compile errors (most of the time not present on Windows) on master only when they make a new release.
Mildly annoying when I try to keep up with the rolling changes, but it is what it is.


----------



## mgp (Jun 21, 2019)

shkhln said:


> It's a fork of a private repo.


Yes, thank you 

malavon, are you aware of https://github.com/UE4-FreeBSD/UE4-FreeBSD (also see https://wiki.unrealengine.com/Compiling_For_FreeBSD )?
Is it possible to somehow build the toolchain for Windows from your repo? Cross-compiling for FreeBSD would be awesome!


----------



## malavon (Jun 21, 2019)

mgp said:


> @malavon, are you aware of https://github.com/UE4-FreeBSD/UE4-FreeBSD (also see https://wiki.unrealengine.com/Compiling_For_FreeBSD )?
> Is it possible to somehow build the toolchain for Windows from your repo? Cross-compiling for FreeBSD would be awesome!


Yes, I'm aware of that. That repo is the one I based my initial port on before redoing it differently in the end. This repo was mainly geared to cross-compilation for FreeBSD on a Windows machine.
It's old and unmaintained though. A few commits still contain changes from that repo.
The wiki is no longer maintained, so as far as I know I can't get this changed. Not that it matters a lot, since cross-compilation is important for Epic to endorse it. See below.


mgp said:


> Is it possible to somehow build the toolchain for Windows from your repo? Cross-compiling for FreeBSD would be awesome!


Nope, cross-compilation isn't supported. Neither from Windows for FreeBSD or from FreeBSD for Windows.
The former should be easier to fix, but I don't have any use for it myself and haven't bothered to support it. I'm not saying it'll never be there, but right now it isn't. I try to go in the right direction as
much as possible when I update versions but until I set up the necessary testing VMs I won't be 'officially' supporting it.
Compiling from *for* Windows for *from* FreeBSD would be a lot harder to do though, since there's no provision in UE4 to do this from another OS like Linux or MacOS.  It'd be cool to have, but way too hard to
attempt this before I have the other direction.


----------



## malavon (Jun 25, 2019)

For the record, there's a new release on the Epic repository (4.22.3) and I have made a branch for it as well.
I won't be building it until at the very earliest next week though, so use at your own peril. After the heat wave has passed, I'll use my processor to warm up the house again 

Edit: 4.22.3 has been released, if there are any issues let me know.

Master still has serious compilation issues for those following that one.


----------



## malavon (Oct 31, 2019)

Ok, I know a few of you are more or less anxiously awaiting a new release and are wondering where it's at right now.
Since it's all been taking much more time than I hoped, I think it's only fair to give you all some information about what's been happening.

For starters, when 4.23.0 launched I intended to rework the third party build system and include a few features that were new to UE4 since 4.22 or 4.23: spatial audio and the chaos destruction framework. Included with that, the demos of these features as well.

When I more or less finished up the 4.23.0 port - still without the new features - and started testing I noticed that Lightmass was no longer working. I assumed because of the exact line of code where it blocked, that it happened because something had been changed in the UDP handling code. I cursed as this definitely wasn't the first time, and then set out to find how that whole piece of code had been changed once again.
I didn't find anything. I couldn't reproduce it outside of UE4 and then decided to go for the long shot: await release 4.23.1 assuming it was very close anyway. That one took a few more weeks than I expected.

Once that version dropped, the release logs mentioned bugs in Lightmass and surely, my (Lightmass) issues were fixed. I'm pretty sure Epic's QA process has a certain hiatus when it comes to testing Linux/Mac related stuff.
Back on track again I set out to fixing bugs where I could find them and get to work on the new features. There are plenty of minor bugs or lack of features - a few of which were carried over from earlier releases - that finally fed me up so much that I just had to fix them.
I won't be going into these just yet, you'll have to wait for an actual release post  Still some work to be done on those as well anyway.
There are a few new ones that appeared as well, breaking PlatformerGame for who-knows-what-reason and I'm going to try and fix these as well before officially releasing. I don't have a habit of letting breaking bugs in something I release.

I am currently being held up on the chaos destruction part. I haven't been able to get it to work and it even crashes the editor in a way I don't think is related to my port. I can't open the ChaosDestructionDemo project either with no obvious reason just yet.
So I intend to verify this using the same UE4 code comparing with the official unmodified sources on a Windows install so that I know it's actually a bug in my port or it's been in that version all along
Of course this also means that I have to install the latest version of Visual Studio (2017 or 2019), since earlier versions are no longer supported by UE4.
Sadly, I have been having serious Windows (registry, libraries etc) issues for years on that installation. I never fixed them since I barely boot it anyway, but this time it came back to bite me. After several days of not being able to get it to work, I'm taking another route.
I'm going to reinstall a fresh Windows install, but I'm currently waiting for some extra SSDs so that I can separate FreeBSD and Windows better. Expect this to be done next week, after which I hopefully can start compilation on a Windows system without glitches.
Once I've done everything I need to to get it running, I'm pretty confident I should be able to ascertain whether or not the port will have a working Chaos Destruction for FreeBSD.

I had so many plans for this release, but am delaying quite a lot to *4.24* now. Getting a working *4.23* out has now become my main priority.

TL;DR: I won't be releasing *4.23.0*, but am working on *4.23.1*. Still a few more bugs to fix, which in turn may mean I'm not releasing in the very near future. It might finally become *4.23.2* if there are more Epic-side bugs that block me.

ps. I'm really bad at keeping people up to date. If in the future you see a long time no port after an official release, just go check the Github to see if a new branch appeared and it basically means I'm trying to get it working again


----------



## kpedersen (Oct 31, 2019)

malavon said:


> Ok, I know a few of you are more or less anxiously awaiting a new release and are wondering where it's at right now.



I often use your latest version when it is released but since you are basically a one man super army on this project I have absolutely no problem with waiting patiently between revisions!

Keeping as up to date as you have been is awesome. Just don't get burned out. Especially if Epic has a lower quality QA process for non-consumer platforms and regressions are inbound .

Is there absolutely zero support from Epic still? I would have thought they might at least donate some SSDs!


----------



## malavon (Oct 31, 2019)

kpedersen said:


> I often use your latest version when it is released but since you are basically a one man super army on this project I have absolutely no problem with waiting patiently between revisions!


When it gets released, there are some worthwhile additions this time.



kpedersen said:


> Is there absolutely zero support from Epic still? I would have thought they might at least donate some SSDs!


To be fair, I never asked any support after the initial port since I was ghosted pretty quickly on the e-mail correspondence. I just assumed that Epic isn't very much interested in this. I might try asking sometime in the future to see if they can do something to help me out.
But in the end, for me it's more like a hobby project anyway. I always assumed I'd get really stuck on something and would have to give up, which up until now never happened.
Also, on the hardware aspect, it's looking like I'm going to have to replace my pc completely. My current build is brought to its knees way too often.
That'll happen somewhere in the next two years probably. I'm currently waiting for the first affordable consumer platform (aka. not Xeon) to support 256Gb of RAM. That's going to be ideal for UE4 development 

While I was typing this post I decided to fill in the Unreal dev grant form. You never know, right  I'll keep you people posted if it ever delivers something.


----------



## malavon (Nov 9, 2019)

Well, it took a long time but here it is in all of its glory: *Unreal Engine 4.23.1*. Like mentioned earlier, *4.23.0* won't be released for FreeBSD because Lightmass is broken in that version.
Chaos Destruction (the 4.23 showcase beta plugin) isn't supported since I couldn't get it to work at all, not even on a native Windows. I'm guessing it may have some issues still. On top of that I discovered that the Windows version warns that an Unreal Engine 4 Editor with Chaos built-in can't open any project that doesn't set the Unique build type.
That's why I haven't enabled it in the FreeBSD port, even though this message isn't displayed on FreeBSD. I'm pretty sure it should be.

On top of that there were quite a lot of issues linking to the system's OpenSSL 1.1 shared object, while later the OpenSSL 1.0 static library was linked into the final binary/lib. What a blast to track those down ... I might've missed a few.
Another thing that was changed in 4.23 is the way platforms are defined. What used to be in code is now contained in configuration files, so I had to re-do some of the porting.

Note that I've only tested this on *FreeBSD 12.0 (amd64)*. I haven't updated to FreeBSD 12.1 yet, though I intend to reinstall my BSD on a new M.2 disk sometime this or next week. I'll test UE4 out when everything is up and running.

*The highlights*
*Audio Spatialization*
The feature that I really wanted to get running eventually made it, cfr. the demo below. It wasn't easy since I couldn't just rely on the built-in audio spatialization features. Libraries that aren't open-source and available for FreeBSD like Steam Audio weren't an option.
The only possibility was Resonance Audio (see below), but in the end I'm really happy about this one. It does its job perfectly and it even sounds better than Steam Audio under Windows.

*Resonance Audio*
Resonance Audio is an open-source Google library for audio spatialization. It can use Intel Embree, which is also open-source. Of course neither of them openly support FreeBSD, but the changes were minimal.
Intel Embree uses the SIMD instructions of modern processors to off-load raycasting from the CPU.
Note that this is the first time that I've included something in the FreeBSD port that isn't available under Linux. You read it: Resonance Audio with Intel Embree isn't configured for Linux. It could be, but it isn't.
Without Embree the audio spatialization just doesn't work, so go brag against your Linux-using friends about that 
On top of that I've upgraded the version of Intel Embree from 2.14.0 (dated 2017/02/09) to 2.17.7, which is the latest 2.X version available right now. This includes quite a lot of bug fixes. That's something even Mac and Windows users don't have!

*Alembic Importer*
Another change I really wanted to make was to provide some way to import meshes, since there is no FBX import. The Alembic importer does just that.
Note that internally it's built pretty ugly, since it relies on the system's OpenEXR version. On other platforms (thus by Epic's or the contributor's design), it uses a hacked way of linking to OpenEXR 2 but at the same time the engine links to OpenEXR 1.
Another reason to keep third party libraries up to date I guess ...

*Breakpad dump_syms segmentation faults*
If you've been building before, you may have experienced segmentation faults on certain modules. That's the dump_syms process crashing, which runs after linking and extract debugging symbols into a separate file.
Since I was really fed up with this - it slows down porting considerably - I decided to track down the issue. Apparently the LibCxx that FreeBSD 12.0 has is either an older one or contains a bug (most likely both).
That's why I've included LLVM's LibCxx and LibCxxabi sources and it's also built along the third party libraries. Breakpad now links against this one.
On Linux this external LibCxx is always used, even for the engine itself but unless I find out that there's a good reason to do so I won't be changing this. The system's c++ lib is generally the way to go.

When I can find the time to look into this I'll create a PR for an update in the FreeBSD sources if it still hasn't changed in 13.0.

*Vulkan*
Vulkan is becoming a recurring theme in my updates. Epic is deprecating OpenGL in favour of Vulkan. I'm not sure when OpenGL will be removed from the codebase entirely, but it'll get less attention for sure now.
So we really need to start begging NVidia to update their drivers  I'll probably create another post for this on the forum.
Of course, I hear AMD is going to release new GPU's soon so I might just switch brands for the first time in decades.
Note that thanks to the solution shkhln has built there's always another path. Native support would still be nice to have though.

Epic has also added a *pop-up* to development builds stating that *OpenGL has been deprecated*. It's an annoying thing, but I've left it in place since it's important to know.

*GPU brand is now recognized*
It might look like a small one - and it kind of is - but it's one more thing that Linux doesn't have. The Linux port will always show the GPU as being a GenericGPU. FreeBSD no longer does.
For NVidia I use the official driver's sysctls and for all others - or if the sysctl doesn't exist - I query the pci bus like `pciconf` does.
`LogInit: OS: FreeBSD 12.0-RELEASE-p8 GENERIC (), CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, GPU: NVIDIA GeForce GTX 770` (yes, I _was_ behind on security updates)
`LogInit: OS: Windows 7 (Service Pack 1), CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, GPU: NVIDIA GeForce GTX 770`

*PhysX compilation issue fix and update to 3.4.2*
Apparently the sources (version 3.4.0, dated early 2017) didn't compile on Clang 6+. I hadn't noticed that the PhysX compilation no longer worked because the cleaning method that I used for third party libraries wasn't adequate.
All *4.22 releases* may have been affected by this, at least when building on FreeBSD 12.
From now on I'll be deleting my entire Engine/Source/Thirdparty before building third party libs to find out about issues before release.
Version *3.4.2* has the necessary fixes and I have upgraded the sources to that version. It means there's one more library that isn't exactly the same as it is on other platforms. Especially one hack of a "fix" has been removed since the original code now asserts that everything is working correctly. I'm pretty sure that at least the FreeBSD part is fine (after blowing up massively earlier), but it's always possible that I've missed an Epic modification to the sources.

*Updated the internal version numbers to the released versions*
This is the first time the engine pretends to be the exact same version as the Windows release. Even though it's not exactly true, it means that it's now possible to compile games and have them communicate with each other effortlessly.
It means that replays can be shared and probably some other goodies as well. It also means that it's easier to share the same build directories under different OS's.
I intend to keep doing this from now on for every release.

*WebM support*
I haven't tested this yet, but the necessary third party libraries have been compiled. I'll test it out somewhere with the next release.
It's meant to play WebM videos from within the engine, next to the other supported formats.

*LTO support and dynamic linker optimization*
LTO (Link-Time Optimization) is now supported for the FreeBSD version. To enable it, add this to the BuildConfiguration.xml:

```
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
    <BuildConfiguration>
        <!--possibly other options here -->
        <bAllowLTCG>true</bAllowLTCG>
        <!--possibly other options here -->
    </BuildConfiguration>
</Configuration>
```
I've built my current editor build with LTO so I'm pretty confident it works well. Expect the gains to be minimal unless it's a monolithic build. Also expect higher memory usage from the linker. You need to have at least one of the llvm ports installed, the base system (at least on 12.0) doesn't have the required llvm-ar binary.
The dynamic linker will now also have to search considerably less libraries due to having less dependencies in the linked shared objects. This will probably speed up the editor's and other modular projects' startup.

*One more demo to rule 'em all*

SpatialAudioTemple (640M) SHA512: 02e16b37466e81c5339fa6dd18a51f55bbacd0a38dfa06d29b90968c4d8bdf430528a81ba06887969a7274206dc9188b3e79f1076841431e03ca1461b0771eca
An already existing demo that Epic modified to showcase the audio spatialization features of UE4. This build is configured to use Google's open-source Resonance Audio library (see above).
Note that this demo also marks the first time something works on FreeBSD which isn't supported on Linux!

*Changes to the build process*

*Additional requirements*
There are now additional requirements in order to use the editor and compile the third party libraries.
`pkg install openexr tbb ispc yasm`

*New third party build system (PortNext)*
The current way of building third party libraries is extremely cumbersome to maintain and over the years I've been wanting to improve on it with much simpler scripts for each library.
I've decided to build something better from scratch and the first fruits of that labour can be found in this version of the port. It's very premature since I've lost too much time tracking down issues with 4.23.0.
The scripts can be found in <UE4ROOT>/Engine/Build/BatchFiles/Linux/PortNext.
As you might assume judging from the name, this system has been loosely inspired by the FreeBSD ports system. I could have called it PortLite or something, but as far as UE4 goes this really is an improvement thus the 'next'.

I might try to get this upstreamed somewhere in the future, although I think Epic is currently exploring the avenue of building third party libraries using the Unreal Build system directly. That's probably better for UE4 anyway.

*Future route*
* I intend to move most, if not all third party builds to PortNext. I'll start with new libraries first.
* I might also include new versions of more libraries, but I'm hesitant about that since many of these may have negative effects on actual builds.
* Master integration has been delayed quite a lot with all the time lost, so that's also going to happen.

*Epilogue*
This has definitely been the most involving release I've done. I've been able to set things right everywhere and several changes are now beyond the Linux version.
It's been a long time I've been able to work on something with at least a modicum of focus and I'm really happy to have done this in almost one straight go.

Special thanks to Frédéric Chopin, Pyotr Ilyich Tchaikovsky, Franz Liszt, Claude Debussy, Antonín Dvořák and all others who helped me through this.

Oh, one more thing. If your project hangs and prints messages like 'Waiting 10 seconds for IO' or something like it, disable "event driven loader" in the project. For some reason this isn't working well.
e.g. PlatformerGame has this issue.


----------



## shkhln (Nov 9, 2019)

malavon said:


> Of course, I hear AMD is going to release new GPU's soon so I might just switch brands for the first time in decades.



Somehow I doubt it. It's AMD after all.


----------



## malavon (Nov 9, 2019)

shkhln said:


> Somehow I doubt it. It's AMD after all.


Still that bad? That's just sad.
Begging to NVidia it is then...


----------



## shkhln (Nov 9, 2019)

Likely will be more stable in a year. (On Linux, that is. Additional time is necessary for FreeBSD porting effort.) I'm a bit of Nvidia fanboy, admittedly, so I don't pay much attention to AMD stuff.


----------



## malavon (Nov 10, 2019)

shkhln said:


> I'm a bit of Nvidia fanboy, admittedly, so I don't pay much attention to AMD stuff.


I myself have just never had any reason to look to other brands since I knew Nvidia had official FreeBSD support. But Vulkan is going to be a dealbreaker for me, so I really hope we can have it from Nvidia as well.
More choice is always better.

I've made a post on this forum to refer to a post I've made on the Nvidia forum, it'd be great if other people could leave something behind there just to indicate people would really like to see this.


----------



## shkhln (Nov 10, 2019)

malavon said:


> I've made a post on this forum to refer to a post I've made on the Nvidia forum, it'd be great if other people could leave something behind there just to indicate people would really like to see this.



Is that even a correct subforum? Unix Graphics is usually the place for these issues. It would likely take a substantially larger amount of complaints to get Nvidia to do something about it, though.

(Which is why it's important to work on improving Wine, Steam integration, controller support and whatever peripheral technologies. We need more people to maintain a proper "complaint presence", so to speak. I can easily count all current FreeBSD Vulkan users on the fingers of one hand. That just doesn't cut it.)


----------



## malavon (Nov 10, 2019)

shkhln said:


> Is that even a correct subforum? Unix Graphics is usually the place for these issues. It would likely take a substantially larger amount of complaints to get Nvidia to do something about it, though.


Somehow I missed that forum, I've asked a mod to move it (though I don't know if there are active mods on that forum).



shkhln said:


> (Which is why it's important to work on improving Wine, Steam integration, controller support and whatever peripheral technologies. We need more people to maintain a proper "complaint presence", so to speak. I can easily count all current FreeBSD Vulkan users on the fingers of one hand. That just doesn't cut it.)


I agree for sure, but no harm in asking twice


----------



## malavon (Nov 15, 2019)

I've checked compatibility of the 4.23.1 release with FreeBSD 12.1 (and thus Clang/LLVM 8) and have found and fixed a few issues.
On top of those I also found one with the nvcore/nvtt library. Make sure to rebuild all third party libs before trying anything.

The modified code will eventually be released with the official 4.23.2 release. In the meantime, there's a branch 4.23.1 that contains all fixes on top of the 4.23.1 release.


----------



## malavon (Jan 14, 2020)

I've released 4.24.1 on GitHub. All useful information can now be found there with the release, along with a link to the readme which explains how to build UE4 on FreeBSD.
There's also a release for 4.24.0, but that one contains only basic getting-to-work code since 4.24.1 came only 9 days after Epic's 4.24.0 release. Backporting 4.24.1 (FreeBSD) improvements would have meant too much work with too few gains.

It took me quite a long time,  in part because I needed to debug the UnrealBuildTool (a .Net application) but monodevelop no longer worked on FreeBSD 12. So after a quite lengthy detour trying to get that rectified, I finally managed to get something working.
A few other (possible) issues were discovered after release.

One of them were OpenGL shaders, but after a lot of investigations I'm starting to think there was file system corruption that may have have impacted this.
In short, after creating a project from the ShooterGame template, the gun of the game would have the wrong shaders (OpenGL only). I can no longer reproduce this, so I consider it a misfortune for now.

Another issue that I noticed was in the SpatialAudioTemple demo. The violin wouldn't start playing the first time it launched, but would the second time around.
If someone else could verify that the demo is working on his end the very first time it'd be very much appreciated.
There's the earlier one compiled for 4.23, and below there's a 4.24 version.

In the future I most likely won't be keeping the master branch up to date anymore. I've been doing this ever since the beginning, but it hasn't really helped me that much. So I'll be sticking to the release branches more often.
If that's the case, I will remove the master branch from my repository altogether in order to not confuse other people.

Links to a few demos, compiled for 4.24 and FreeBSD 12.1 (amd64):

ArchVizInterior (demo mainly meant to showcase raytracing, which is only supported on DX12)
InfiltratorDemo
ShooterGame
SpatialAudioTemple


----------



## malavon (Feb 26, 2020)

Version 4.24.2 is on Github since yesterday, of course a few hours later Epic had to release a new version already 
There's a catch though, my GTX 770 has stopped working and I no longer have a decent platform to test on. Right now I'm on my built-in Intel video, but that won't run UE4 stuff.
On top of that I don't really want to buy a new videocard: I don't know what NVIDIA is going to do with Vulkan, AMD might release cards that support raytracing and Intel is coming to GPU land with discrete GPUs.
I'll keep you posted if something changes.


----------



## shkhln (Feb 27, 2020)

Well, if that helps you with purchasing decisions, with recent Nvidia drivers I'm unable to run q2rtx neither with my shim nor under Linuxulator. Other than ray tracing extensions, Vulkan seems to work fine.



malavon said:


> AMD might release cards that support raytracing



They will likely start with a pure software implementation.



malavon said:


> Intel is coming to GPU land with discrete GPUs.



Meh.


----------



## aimeec1995 (Feb 27, 2020)

Could retail UE4 game be made to work on freebsd with this port of the engine?


----------



## shkhln (Feb 27, 2020)

aimeec1995 said:


> Could retail UE4 game be made to work on freebsd with this port of the engine?



No, it's only useful to developers.


----------



## kpedersen (Feb 27, 2020)

aimeec1995 said:


> Could retail UE4 game be made to work on freebsd with this port of the engine?



Unreal Tournament 4 provides access to the source for the public (after signup) so in theory could be a retail game that is portable to FreeBSD. They could close up the source nearer to beta however.

I think development may have stalled whilst Epic milk Fortnight however


----------



## malavon (Feb 27, 2020)

aimeec1995 said:


> Could retail UE4 game be made to work on freebsd with this port of the engine?





shkhln said:


> No, it's only useful to developers.


Well, in theory any UE4 game could be compiled by its development team to produce native binaries for FreeBSD. In practice this would require a UE4.19 or higher and there would probably be some bugs to work out as well.
But indeed, an already compiled game wouldn't be able to be ported easily.


kpedersen said:


> I think development may have stalled whilst Epic milk Fortnight however


From what I once gathered though take this with a heap of salt, UT4 was supposed to be the milk cow for Epic with skins, maps, gamemodes etc being sold on the marketplace. 
Although never official, this is most likely why UT4 was abandoned after Fortnight took off. Also, the (external) development team was removed from the project.


----------



## malavon (Mar 11, 2020)

4.24.3 is on GitHub now as well. It has been tested on my poor little embedded graphics card (at about 2 fps).


----------



## kpedersen (Mar 11, 2020)

malavon said:


> It has been tested on my poor little embedded graphics card (at about 2 fps).



Haha. I have a Quadro k440 and a GeForce 4 (AGP) that I am probably never going to use if you would like me to send it to you. Still not massively powerful (and as I am sure you already know, they both require a very poor quality driver to be functional).

As always, thanks for your work!

Edit: I think I have a RadeonHD 6xxx too. If you have time to write an ~OpenGL 2.x renderer for UE4? 

Edit2: Old shite is kinda my thing so I possibly am not the most useful hardware hoarder for this XD


----------



## malavon (Mar 13, 2020)

Yeah, I have some old stuff as well but nothing usable or available to use. Doesn't really matter atm though, I'll get a new one somewhere in the future 
Timing might be wrong for the 4.25 release though, it might be the first range I'll miss.


----------



## malavon (Mar 18, 2020)

Apparently my 770 decided to come back alive, no idea why it didn't work before (tested in several computers etc).
So if it keeps up, nothing has changed and 4.25 will be ported and tested as well as other versions, barring the usual risks.


----------



## NuLL3rr0r (Mar 29, 2020)

Again, I would like to thank you once more for the consistent effort you are putting into this port. I am considering moving away from Gentoo on desktop as I am very satisfied with my experiments with ZFS (Linux does support ZFS, but well I liked the easy setup on FreeBSD), and your port is one reason I can easily go back to FreeBSD on the desktop without losing UE4. But, I am considering a longterm decision. So, have you ever considered backporting your changes to upstream?

I know EPIC is painfully slow at accepting pull requests on GitHub. But, I would like to know if it ever happened and they were receptive or not.


----------



## malavon (Apr 3, 2020)

NuLL3rr0r said:


> But, I am considering a longterm decision.


If I die of a certain virus somewhere in the next few weeks/months/years there will no longer be a FreeBSD UE4 port. The only possible option would be to keep the port up to date yourself.
It's actually surprisingly easy, it just takes quite some time to track down where things have broken compared to the previous version.

I know it's probably not what you wanted to hear, I'm just trying to paint a realistic picture here. One-man efforts can easily stop due to a multitude of factors. If you're really dependent on UE4, you'd be making yourself dependent on me and my porting/programming skills. Also, I have no idea if I could call my port production quality since I don't use it myself except for some tinkering/experiments. On top of that, one day there may be some required closed source library which I can't circument like I did with the FBX importer. At that moment it'll stop because I have no other option.


NuLL3rr0r said:


> So, have you ever considered backporting your changes to upstream?


Yes  but see below.


NuLL3rr0r said:


> I know EPIC is painfully slow at accepting pull requests on GitHub. But, I would like to know if it ever happened and they were receptive or not.


Well, back in 2014 there seemed to be some interest in this from Epic's side. See https://www.unrealengine.com/en-US/blog/unreal-engine-4-and-linux. This changed later on or was a bit of a marketing thing.
As you may well know I originally based my work on the work done by mdcasey in this GitHub repo sometime in 2016. He had been trying to get FreeBSD support mainlined in a few PR's, but eventually abandoned it when it looked like it wasn't going anywhere.


> Unfortunately, we will not be able to maintain FreeBSD support in the engine, but I suggest creating a fork dedicated purely to FreeBSD support and adding the link to it to the public wiki.



I myself have also contacted this Epic developer privately via e-mail on this back in 2018 and I basically got the same response. Epic isn't interested in having this code in their repository, since they won't be maintaining it themselves. In fact, they've recently begun making changes to eliminate all platforms they no longer want to support from the official repository. One example of this is HTML5 support. Sidenote: this now also makes it impractical for me to support HTML5 on FreeBSD, because I'd have to support two separate ports.
More recently I've also contacted Epic to know if they'd be interested in more or less making FreeBSD official through their dev grant system by basically throwing some money in my direction, but until today I haven't received anything back except a "we're looking into it" type of response. If they were really interested in FreeBSD support, I'm pretty sure they'd jump on the opportunity of having someone do it for far less than the cost of the actual maintenance.

Apparently the UE4 Wiki has been taken offline by Epic. I think there was some more information on there, but don't know for sure anymore.

TL;DR: One-man projects are always a risk and Epic isn't really interested.


----------



## NuLL3rr0r (Apr 4, 2020)

Thank very much for the detailed clarification and the info. Well, I expected to hear they are not interested (as corporates number one priority is the market and money). But, by following the links you have posted, I noticed they accept smaller pull requests which are good news.

Anyway, I am considering moving back to FreeBSD on Desktop (I use it regularly on servers). I am a gamedev who has been away from game development for almost two years now (I did three games in UDK and UE4 in the past). I guess I'll get an extra SSD/NVME drive and try to experiment with your port to see where it goes before I make the permanent switch. If I am successful with it, who knows, maybe I join you in this effort which will be a good refresher for me.

Thanks again for the response and for keeping this port up to date.


----------



## kpedersen (Apr 4, 2020)

malavon said:


> One-man projects are always a risk.



I certainly see what you are saying but I also know of a lot of one-man projects that have outlived commercial support. Commercial companies are like little fat children, they lose interest and ditch their toys XD.

Even though I don't know who you are (other than recognising the name from IWD2), when it comes to technical competency, I actually have less confidence in game development companies haha.

What is fantastic about your work is that when Epic goes bankrupt or makes an Unreal Engine 5 and ultimately ditches UE4, we know that we still have your work on UE4 available to us and we only need to maintain it from breakages from FreeBSD itself .

If it was withheld source without an official FreeBSD port like Unity (the prosumer toy version of the game engine world), any commitments a developer has made up until that point would have been wasted when the company goes bankrupt. I feel because of your efforts we are in a much better position here.

I am more than happy to use old software (even you first port!), so I just hope you do not feel pressured to keep in sync with upstream.


----------



## malavon (Apr 4, 2020)

NuLL3rr0r said:


> I am a gamedev who has been away from game development for almost two years now


If you'd be interested in getting your most recent project (Reminiscence) running on FreeBSD hit me up. I'll guide you through it as good as I can. Having someone who runs a "real" project on FreeBSD would be great to check if the FreeBSD port doesn't have any more bottlenecks than Linux etc.


kpedersen said:


> Even though I don't know who you are (other than recognising the name from IWD2), when it comes to technical competency, I actually have less confidence in game development companies haha.


Ah, the name. Yes, it's originally from IWD (not 2 though). It's also the name of my first and most precious AD&D2 character, a true neutral necromancer. I don't play anymore though.
As far as technical skills go, I really don't compare to what the (engine dev) guys at Epic are really doing. I couldn't write this whole thing from scratch. The only thing I can say is that if I tried it'd be cleaner and unit tested code at least 


kpedersen said:


> What is fantastic about your work is that when Epic goes bankrupt or makes an Unreal Engine 5 and ultimately ditches UE4, we know that we still have your work on UE4 available to us and we only need to maintain it from breakages from FreeBSD itself .


I'm not entirely sure if the EULA (aka the license) doesn't make it possible for Epic to remove all forks in existence and just retract the license they've given us to maintain them. I skimmed through it and I think they can't do this legally, but who knows.
You are right though, up to now some things have been broken when OpenSSL was updated, Clang versions broke some things etc but nothing really major.


kpedersen said:


> I am more than happy to use old software (even you first port!), so I just hope you do not feel pressured to keep in sync with upstream.


Not really, I still kinda feel like it's fun to get it working. If I don't feel like working on it, I just don't. It's one of the benefits of doing this purely for fun. The novelty has worn off though, I rarely get excited when a new version is released.
In fact I've been awaiting the release of a version which has a non-beta of Chaos because it's something new and shiny, but it would appear this won't be the case for UE4.25. More focus has gone to archviz stuff which I assume is also a huge market for Epic.


----------



## NuLL3rr0r (Apr 5, 2020)

> If you'd be interested in getting your most recent project (Reminiscence) running on FreeBSD hit me up. I'll guide you through it as good as I can. Having someone who runs a "real" project on FreeBSD would be great to check if the FreeBSD port doesn't have any more bottlenecks than Linux etc.



Sure, I definitely will test it out.

My current hardware is Asus ROG G752VS-XB78K Overclocked Edition. The touchpad (ELANTECH 1200) was a hassle with Linux until 4.19. I even risked and ran this binary to make it work with Linux:



			https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1653456/+attachment/4955397/+files/WinIAP_Fix%20Touchpad.zip
		






__





						Bug #1643242 “ASUS ROG G752 ELAN1203:00 04F3:301E Touchpad detec...” : Bugs : linux package : Ubuntu
					

Model: Laptop ASUS ROG G752 VM dual boot with Windows 10; security boot disabled; touchpad activated in uefi  Touchpad: Name "ELAN1203:00 04F3:301E Touchpad" (see dmesg.txt)  OS: Ubuntu 4.8.0-27.29-generic 4.8.1 (also tried 4.4.0 and some others)  Symptoms: - Touchpad seems to be correctly...




					bugs.launchpad.net
				








__





						Bug #1653456 “ASUS G752VS: Touchpad and Fn keys not working (Ubu...” : Bugs : linux package : Ubuntu
					

Non-Optimus laptop ASUS G752VS-GC063D-CST256. 17.3" FHD LED 1920x1080, Intel Core i7-6700HQ (3.50Ghz), 16GB DDR4, 256GB M.2 NVMe SSD + 1TB HDD 7200rpm, DVDRW-DL, Nvidia GTX1070 8GB GDDR5, Wifi 802.11ac+Bluetooth 4.1 (Dual band) 2*2, Gb LAN, HDMI, mDP, Intel WiDi, USB3.0 x4, USB3.1-Type C(Gen2)...




					bugs.launchpad.net
				




So, I decided to download FuryBSD 12.1 to try a LiveCD and see if it works before installing FreeBSD. Well, it didn't work. But, I found a link to a patch file here: https://reviews.freebsd.org/D16698 and unfortunately, it requires approval after signing up. I've been waiting a few days. So, once I get that fixed, I'll try out your port.


----------



## malavon (Jul 3, 2020)

I've released 4.25.1. There won't be a 4.25.0 release, that one was broken due to UE-92806 (even though the description doesn't really give it away). 
Originally I had added ccache to this release, but in the end it seemed that the UE build system didn't profit from it. I might revisit that in the long run again though because I could really use it myself to speed up things.

There isn't much news to share, except that Epic has added something that changed the way the editor works: after creating a new project, the project is not opened in the editor but in the default file browser. 
Compilation of the new project needs to be done manually as well first (at least for C++ projects), it's not kicked of by the editor anymore.

I'm going to try and revisit my wine experimentations until next version is released.


----------



## Jose (Jul 3, 2020)

All your Github links 404 for me. There's no UnrealEngine listed under your repositories. Is it not public, or did Epic get ya?


----------



## kpedersen (Jul 3, 2020)

Jose said:


> All your Github links 404 for me. There's no UnrealEngine listed under your repositories. Is it not public, or did Epic get ya?



It is a private repo upstream so any forks on GitHub are too. So this means you need to sign up to the UE4 repo first before you will see any of them.

Some instructions are here: https://www.unrealengine.com/en-US/ue4-on-github


----------



## Jose (Jul 3, 2020)

kpedersen said:


> Some instructions are here: https://www.unrealengine.com/en-US/ue4-on-github


I'm losing interest rapidly.


----------



## kpedersen (Jul 3, 2020)

Jose said:


> I'm losing interest rapidly.



Haha. Actually I was wondering myself why the process hasn't been made a little easier now that Microsoft has taken over GitHub. I would have thought they would have turbo charged the "Paid for code access" API the first second they could.

That said, it is certainly worth it. Malavon has done a great job at porting it. It works well and I would say that it is very production ready. If it wasn't for licensing issues (and logistics) it would have been a great addition to the ports in its current state.


----------



## Jose (Jul 3, 2020)

I'm not going to approach a for-profit company on bended knee and beg them to let me work for them for free. Especially not if the results of my labor are not going to be shared freely. I get compensated handsomely for doing that sort of thing.


----------



## kpedersen (Jul 3, 2020)

Jose said:


> Especially not if the results of my labor are not going to be shared freely. I get compensated handsomely for doing that sort of thing.



Yep, I do understand that exactly. You could start an open-source UE4 game and share that with people but the underlying engine looks unlikely to ever be free.

I would say that Epic's UE4 is doing better than (err) Unity's Unity in that when Epic goes bust, you can keep your project going (unlike Unity which will slowly bitrot into legacy platforms only just like Adobe Flash).

But yes, for my own passion projects, I am like you. Stay light and free. Only recently I wrote a shadow baking tool for my own games just to avoid proprietary (or very large) software which unfortunately can be relatively short lived and restricting.

But again... in 30 years, I doubt anyone will be around to sue me for releasing a UE4 based game open-source. If they are, I will simply do it as one last laugh on my death bed! XD


----------



## malavon (Jul 5, 2020)

Another thing I forgot to mention. Unreal Engine 5 has been set for 2021, no idea what the future for UE4 will be.



Jose said:


> I'm not going to approach a for-profit company on bended knee and beg them to let me work for them for free. Especially not if the results of my labor are not going to be shared freely. I get compensated handsomely for doing that sort of thing.


Not sure what the deal is really, after all it's just registering an account with Epic and linking your GitHub account. But then again I'm quite pragmatic. Also: free games on that Epic account weekly.
If that's the only thing to do to gain access of tens or hundreds millions of $/€ worth of code, I think it's a pretty good trade off. It's even less difficult than signing an NDA, which is the least to be expected in such a situation really.
Working for free is definitely something you don't have to do. If you're referring to what I do, I see it as giving something to the FreeBSD community, not to Epic.


kpedersen said:


> But again... in 30 years, I doubt anyone will be around to sue me for releasing a UE4 based game open-source


You can legally do that today already, you just can't include a copy of the engine. No fee is required if there's no revenue. All code you write remains yours.


----------



## kpedersen (Jul 5, 2020)

malavon said:


> You can legally do that today already, you just can't include a copy of the engine.



Yes of course. However imagine in 30 years I did that. The source would be useless to someone without the rest of the (long since obsolete version) engine. It is not like my code will be able to instantly work on the then current version of Unreal Engine 8. If going by UE3 -> 4 then it most certainly won't.
The potential players would need to track down a specific version of the engine which has no guarantee to still be available.

So yes, I quite possibly would release it all together and hope Epic doesn't wake from the care home to sue me haha. That said, I am making a fairly large assumption that UE4 will still be working (or can even be compiled!) on our magical quantum computers of the year 2050 (more like crappy consumer phones connected to expensive subscription grids).

So, if in 2050 you see a tonne of random LEGO games (though none are UE4 based and full of bitrot!) appear on the torrents along with source code... it may or may not be me XD


----------



## Jose (Jul 5, 2020)

I really appreciate it when people work Freebsd because it's my chosen platform. I'm grateful for what you do even if I'm not likely to ever use it. Thank you for making it possible to work with the UE engine on Freebsd.

Working like this is just not for me. Having to keep private forks of a private repository so that my work is not gratuitously erased upstream because they don't feel like "supporting" it is not something I want to do. The worst part for me is that no one can pick up my work and build on it because it's invisible to most people.


----------



## malavon (Jul 6, 2020)

kpedersen said:


> The potential players would need to track down a specific version of the engine which has no guarantee to still be available.


That would definitely be an issue. There's always the chance that in the long run - think ten or twenty years - Epic releases UE4 as real free unencumbered open-source. It all depends on how useful they still deem it I guess.
But like you said, the chances that we'd even be able to compile and run might be rather low anyway.
What I did however mean is that if you want to start an open-source game, there's nothing to hold you back legally right now. Feel free to code something and release it to all of us 



Jose said:


> The worst part for me is that no one can pick up my work and build on it because it's invisible to most people.


It's not really invisible, is it? It's only invisible to people not interested in the Unreal Engine. The ones who want to download it, eventually still end up with the right instructions anyway.


----------



## kpedersen (Jul 6, 2020)

malavon said:


> there's nothing to hold you back legally right now. Feel free to code something and release it to all of us



There should be. It should be illegal for me to make games and subject people to the torture XD

These are two such classics!






						Steam Community :: Error
					






					steamcommunity.com
				








						Steam Community :: Error
					






					steamcommunity.com
				




My issue is that everything I make in terms of art is painful on the eyes. I accept this, and I revel in it 
No matter how advanced and beautiful UE4 gets. There is probably no helping me haha.

So instead I stick to non-gaming / 3D simulation contracts... Safer for everyone.


----------



## malavon (Jul 7, 2020)

kpedersen said:


> My issue is that everything I make in terms of art is painful on the eyes. I accept this, and I revel in it


I share that with you, couldn't make something beautiful if my life depended on it.


----------



## kpedersen (Jul 7, 2020)

That said, I am fairly excited because I am nearing completion of my raytracing lightmap generator (so I can avoid the scrappyness that is Blender scripting as well as constantly replacing code as functionality is dropped).

My theory is if I make everything helplessly dark, then people won't be able to see my horrible artwork


----------



## skeletonboss12 (Jul 7, 2020)

Are there any tunables I should try tweaking for gaming in freebsd? My workstation locks up quite a bit when i am doing it and I have plenty of free resources.


----------



## malavon (Jul 8, 2020)

skeletonboss12 said:


> Are there any tunables I should try tweaking for gaming in freebsd? My workstation locks up quite a bit when i am doing it and I have plenty of free resources.


You'll probably be better off making your own topic, only people interested in the UE4 port are in this topic. Also try to be a bit more detailed (which game, emulated or native, what hardware etc).


----------



## skeletonboss12 (Jul 13, 2020)

A private repo. 
Well, there goes my interest.


----------



## malavon (Jul 28, 2020)

And yet another release, 4.25.2. Have fun with it!


----------



## malavon (Jul 30, 2020)

And one day after I released 4.25.2, Epic decided to release 4.25.3  So that's been released for FreeBSD now as well.


----------



## nmcunl (Sep 5, 2020)

Dear malavon the github links to the FreeBSD port aren't working  is the repo not available any more?


----------



## Jose (Sep 5, 2020)

nmcunl said:


> Dear malavon the github links to the FreeBSD port aren't working  is the repo not available any more?





kpedersen said:


> It is a private repo upstream so any forks on GitHub are too. So this means you need to sign up to the UE4 repo first before you will see any of them.
> 
> Some instructions are here: https://www.unrealengine.com/en-US/ue4-on-github


----------



## nmcunl (Sep 5, 2020)

Thank you so much kpedersen  <3


----------



## malavon (Oct 24, 2020)

A bit unexpectedly Epic released a new version in the 4.25 branch even though they're working hard on 4.26. 4.25.4 is on GitHub now.
Not sure what they're doing with future versions, the roadmap mentions using the GPU for lightmass calculations. I hope they won't remove the CPU version since we don't have OpenCL/CUDA for NVIDIA cards on FreeBSD.


----------



## kpedersen (Oct 24, 2020)

malavon said:


> I hope they won't remove the CPU version since we don't have OpenCL/CUDA for NVIDIA cards on FreeBSD.



Eeek. That would be a shame. Perhaps there are still a number of platforms out there that Epic want to support without CUDA implementations? Mobile / console?



nmcunl said:


> Thank you so much kpedersen  <3



Ah I missed this. No worries. But the real person to thank here is malavon. He is doing the entire port single handedly. I am just a forum lurker


----------



## malavon (Oct 25, 2020)

kpedersen said:


> Eeek. That would be a shame. Perhaps there are still a number of platforms out there that Epic want to support without CUDA implementations? Mobile / console?


Well, lightmass runs during the build/bake process, so what matters is the development system (Windows/Linux for Android, MacOS for IOS). I'm pretty sure they can change this without impacting their actual development platforms.
We'll see what happens.

Sidenote, it appears that Chaos is production-ready for 4.26. I hope Epic decide to release their destruction rail demo so it can be compiled for FreeBSD as well.


----------



## kpedersen (Oct 25, 2020)

malavon said:


> Well, lightmass runs during the build/bake process, so what matters is the development system



Ah yes; good point!



malavon said:


> Sidenote, it appears that Chaos is production-ready for 4.26. I hope Epic decide to release their destruction rail demo so it can be compiled for FreeBSD as well.



That is looking pretty awesome. It makes me shudder to think how much complexity there is in the engine now!

Do you see any potential work being done to speed up the iterative build times? Any stubs to Cling? Or perhaps even some hint that the Python layer will be within the engine itself rather than just the editor?


----------



## malavon (Oct 25, 2020)

kpedersen said:


> That is looking pretty awesome. It makes me shudder to think how much complexity there is in the engine now!


I haven't looked into it very much, but since it's encapsulated inside the engine it's probably easier than it used to be with PhysX. Complexity may have gone up, but UE4's modularity helps separating everything nicely.


kpedersen said:


> Do you see any potential work being done to speed up the iterative build times? Any stubs to Cling? Or perhaps even some hint that the Python layer will be within the engine itself rather than just the editor?


Python is still in experimental status, but as far as I see it's possible to use Python (code-wise that is) aside from the editor. I haven't tried it though, so there might actually be issues there. But it's not documented as such.
Note that Python support in FreeBSD is a bit sketchy right now, it requires python 2 instead of the 3 branch.
Any other script-like thingies might be there but I generally only see the parts of the engine that break  It shouldn't be too hard to implement though, take a look at the ScriptPlugin plugin. It's an example how to add scripting languages.


----------



## malavon (Apr 16, 2021)

Ok, since there are 2 bugfix releases for the 4.26 branch already and I haven't released any FreeBSD versions yet, a quick look into why that is.
The 4.26 branch is the first where Vulkan is the only option available to Linux users (and thus FreeBSD). So logically I've assumed that that would make my life easier, since there's only one RHI to test anymore. There were a few other things that took some time, but nothing earth shattering. However, what looked like a very simple maintenance release has turned into a nightmare. Vulkan used to work before but I can't get it to work with the 4.26 branch - which uses different initialization code - at all. I assume it does work on Linux (haven't tested it), but it just won't work on FreeBSD. Maybe there's some obscure issue with FreeBSD 12.2 (previous releases were on 12.0 & 12.1), but I highly doubt that.

Each UE4 release I keep hoping that there was an issue that has been resolved. Twice I have been disappointed. I've done everything I can to find out if there's an issue with my system, but all other vulkan software works without issues. When NVIDIA released their beta driver, I hoped that maybe there was a very obscure error in the nvshim that I use for Vulkan support. I tried an entire month to get Vulkan working and failed each and every time.

For the record, I don't know anything about Vulkan programming. I can't just look at the initialization code and spot what's wrong. The only way right now that I think I can try and solve this is by starting the 'random change/delete' method. Just keep deleting code until it works and then figure out why. The other alternative is to reimplement the whole init code with the Vulkan examples and Khronos documentation and then work from there.
Both are however difficult, with code spread over many places inside the engine. Even the vulkan headers are available in three different versions throughout the codebase, which makes debugging even harder.

If someone wants to try and have a stab at it, please do. All releases are on my github in separate branches for each release, without any major modifications to the Vulkan stuff. Everything compiles on FreeBSD and I'm pretty certain it'll work as it is, except for the Vulkan part that is. At the very least this could tell me whether or not there is maybe a local issue that I'm not seeing.

Anyway, I do hope to take another stab at it sometime but I'm really despairing each time I just open up the code. I feel like I'm missing something incredibly obvious as this has happened before (non-vulkan), but it's the first time I'm not having any luck finding it.
Hope to get back to you all one day with better news though. Right now I am however busy with another (commercial) project, of course not UE4 related and I don't expect to find much time for this for several more months.


----------



## SteamBSD (Apr 17, 2021)

Use GODOT: https://store.steampowered.com/app/404790/Godot_Engine/?l=russian&curator_clanid=26020463





--- SteamBSD © is FREE operating system.
YouTube: https://www.youtube.com/channel/UC8wwRY8yGWiJ-bIQlK0wvUA/videos
Site (download ISO/IMG): https://lpros.blogspot.com
Github (internet installer): https://github.com/steambsd/os
Email: steambsd@gmail.com


----------



## kpedersen (May 7, 2021)

Hi malavon,

Hope things are going well with your other commercial project!

I have been faffing about with the internals of UE4 for a slightly unrelated project (ugh Hololens!). I then started pondering how far we could go without Vulkan support. I.e you mention that it does all still compile other than the Vulkan parts.

This is more of a thought exercise because I am rarely in a rush for updates personally (I often lag behind many versions!). Though if one was necessary, I suppose we could keep going like normal and just neglect the Vulkan parts until a day when Epic will likely clean them up anyway and then perhaps it could be re-considered.

My reasoning is that I have had to hack in a few parts in DirectX for the UWP / Hololens stuff. I encountered some Vulkan parts that I don't have much of a clue on and don't really intend to learn anyway... so they might as well not be there


----------



## malavon (Jun 14, 2021)

Ok, a small update. When Epic released the 5.0 EA* version I just had to see if I could get that one working and couldn't stop until I had at least found the Vulkan issue.

Let's start with the good news: I've been able to find where it all went wrong with the setup and I barely dare say it honestly. A bunch of assumptions that turned out bad and me looking at all the wrong places caused this. I'll explain more when I finally manage to create something that's working without issues.
Now the bad news along with the better news: the current implementation of the memory management code seems to have some issues where it goes out of (video) memory badly. A 4Gb card just no longer runs the InfiltratorDemo - which was originally shown on a GTX 680 with 4Gb VRAM - and it crashes on basically all other projects. However ... I've been re-implementing it by integrating an open source Vulkan Memory Manager and have had quite good success with it. I'm still trying to integrate it in all code without crashing but it looks like it should be possible. Most likely I'll be releasing - eventually - with this code instead of Epic's. That is unless Epic releases a 4.26.3 with fixed Vulkan code which would render all of my work useless :-D

* Unreal Engine 5 could actually work on FreeBSD as it is right now, I haven't seen much that could really stop me. There is one thing though: the build system no longer uses mono, instead it uses dotnet core (3.0) and apparently the included C# compiler segfaults when running under the Linuxulator. I think we're missing some kernel calls or libraries to run the more recent dotnet versions - as also evidenced that they're not in ports. I didn't look into it too much so I might eventually find out that it's an easy fix.


----------



## kpedersen (Jun 14, 2021)

Hmm, interesting update. It looks like there is some substantial thought and implementation going into this. Nice work!

Quite interested in trying out UE5. I know it isn't really quite as extreme as UE3 <-> UE4 but since it is a major release I thought I should try it out.

A little disappointing they are still using mono for the build system, ubt, etc. Surely an interpreted language (i.e Python) or a light language (Lua) would be so much less of a faff. The technology behind Mono (and Java) is very unportable contrary to popular belief. Since all it is really doing is generating text, it is almost irritating it needs a complex VM requiring "exotic" kernel calls.
(Perhaps I am just bitter because I needed the entirety of Mono installed just to extract a file in my other project )


----------



## malavon (Jun 14, 2021)

kpedersen said:


> Quite interested in trying out UE5. I know it isn't really quite as extreme as UE3 <-> UE4 but since it is a major release I thought I should try it out.


I like it, or at least the bits I've seen up to now. It's still UE4, but with a few nice extra additions like Nanite. I really like the UI though, much cleaner than the UE4 one. Except for the Dotnet Core part it should be possible to port I think.


kpedersen said:


> A little disappointing they are still using mono for the build system, ubt, etc. Surely an interpreted language (i.e Python) or a light language (Lua) would be so much less of a faff. The technology behind Mono (and Java) is very unportable contrary to popular belief.


It's even worse, it's the Microsoft version of it. At least Mono runs natively, Dotnet Core runs in Linux emulation mode. I really would have preferred an existing build system tbh. CMake would make a great addition to it, especially if Epic were to implement their Unity (I always chuckle when I see that term) build natively under CMake. I'm sure that 90% of the modules could easily build with CMake, it's just that the remaining 10% is full of cyclic dependencies. Then again that's a consequence of having a build system that allows bad coding habits to work.


----------



## malavon (Apr 21, 2022)

Ok, it has been a long time replying in this post...but I've recently released version 4.26.2.  Release notes etc can also be found over there. They're quite extensive this time and contain some interesting information.
I have added a "discussion" on GitHub in case anyone might want to list up some things that work or don't work. Compilation issues etc can also be addressed over there if need be.
Note that I've released this one using a script, so if you were following the repo you may have seen a gazillion different release pop up. Just testing the script...

There are many reasons it took me this long, but most of them aren't very relevant and considering the timeframe I've forgotten them anyway. One however is interesting:
I'm trying to "prep" the UE4 releases to be as resilient to (FreeBSD) changes as possible. Ever since epic released UE5 previews I've known that UE4 wouldn't last very long anymore. And since every FreeBSD upgrade (with new LLVM etc) breaks my UE4 ports, I needed to make some changes. So, that's pretty much what I've been doing for the last 4.x releases. Also note that I chose no longer to port intermediate versions (e.g. 4.26.1), hoping it would gain me some time.
Part of getting it a bit more resilient is also to make it more usable. I've ported the USD importer and it's also possible to use the glTF format for models. Bot importers seem to work well.

On another note: there is no more custom Vulkan implementation either. Ever since the new NVIDIA driver (470.10x) I've no longer experienced any issues with Vulkan going out-of-memory on the InfiltratorDemo. I'm not 100% sure if that's the cause, it may have something to do with other fixes I've done. Anyway, I'm certain there is no more need for a custom implementation, so that's a win for sure.

There's a lot of nice stuff in this release and I'm quite happy I've been able to finish it. The next and last UE4 release (4.27.2) will focus on getting it even more usable on FreeBSD.

And one last thing I've been wanting to say, but couldn't really without getting up to date with the releases a bit: I've been granted a small Unreal Dev Grant for keeping the FreeBSD port running. It's nice to know Epic at least acknowledges the port.
Maybe I should start adding the Dev Grant logo on the UE4Editor splash screen


----------

