[{"content":"Some months ago I vibe-coded a browser-based Unreal Engine asset viewer and forgot to publish or mention it anywhere. I thought it was pretty interesting to see how the bytes are actually laid out on the file.\nYou can try it out here. It contains some sample assets as well so you don\u0026rsquo;t need any .uasset ready on your end.\nHere is what it looks like:\nIt parses .uasset and .umap files, shows you the raw bytes, parses values and describes what the fields mean.\nIt\u0026rsquo;s purely client-side: The file never leaves your machine and nothing is uploaded anywhere.\nIt should work for most UE 5.x assets and some 4.x assets, with some luck!\nFeatures Hex viewer: Every byte of the file, with color-coded ranges marking what each region actually is. Canvas-based with virtual scrolling, so large assets stay smooth. Annotations panel: A collapsible tree of named byte ranges, each with its size, name, and parsed value. This is the part that replaces the manual byte-counting. Summary panel: Asset class, engine version, package path, and the embedded thumbnail if the asset has one. Minimap: A proportional overview of the whole file with a viewport indicator, which turns out to be a surprisingly good way to get a feel for how an asset is laid out. Search: By hex bytes, ASCII text, byte address, or annotation name (Ctrl+F / Ctrl+G). Context menus: Copy the address, the raw bytes, or the ASCII text of any annotated region, and scroll the other views to match where you clicked Limitations Properties: It should display some types of serialized property values but not all, and also won\u0026rsquo;t handle nested structs super well. It\u0026rsquo;s open source and hosted on Github here.\nCheers!\n","permalink":"https://1danielcoelho.github.io/uasset-viewer/","summary":"\u003cp\u003eSome months ago I vibe-coded a browser-based Unreal Engine asset viewer and forgot to publish or mention it anywhere. I thought it was pretty interesting to see how the bytes are actually laid out on the file.\u003c/p\u003e\n\u003cp\u003eYou can try it out \u003cstrong\u003e\u003ca href=\"https://1danielcoelho.github.io/uassets/\"\u003ehere\u003c/a\u003e\u003c/strong\u003e. It contains some sample assets as well so you don\u0026rsquo;t need any .uasset ready on your end.\u003c/p\u003e","title":"Browser-based Unreal Engine UAsset viewer"},{"content":"A good part of my day consits of waiting for Visual Studio (VS) to respond. I work with Unreal Engine (UE) on Windows and VS seems to be the recommended IDE for it, even though it\u0026rsquo;s not really required by UE or its build system in any way. There are many other IDE choices out there, but I tend to use VSCode for other projects and note-taking, so I wanted to try exclusively using it for UE as well, and this is what I got.\nI think this post could be useful even if you don\u0026rsquo;t plan on doing exactly this, as we\u0026rsquo;ll end up setting up, building, cooking, packaging, and etc. manually \u0026ldquo;from the command line\u0026rdquo;, and you can probably use some of that in order to create/enhance your own setup.\nThis post goes over the main aspects of the setup first, and at the end ties them all together into a clear-cut step-by-step, which sort of works as a TL;DR too.\nVSCode workspace First of all, it\u0026rsquo;s handy to have some workspace-specific settings, so I recommend setting up an actual VSCode workspace. I use a single workspace directly on the UE install root (so next to the GenerateProjectFiles.bat and Setup.bat scripts), and add the folders for each individual UE project. The VSCode workspace JSON file looks like this:\n{ \u0026#34;folders\u0026#34;: [ { \u0026#34;path\u0026#34;: \u0026#34;.\u0026#34; }, { \u0026#34;path\u0026#34;: \u0026#34;F:/MyProject/\u0026#34; } ], \u0026#34;settings\u0026#34;: { \u0026#34;editor.tabSize\u0026#34;: 4, \u0026#34;editor.insertSpaces\u0026#34;: false, \u0026#34;editor.glyphMargin\u0026#34;: true, \u0026#34;git.enabled\u0026#34;: false, \u0026#34;files.exclude\u0026#34;: { \u0026#34;**/.cache\u0026#34;: true }, \u0026#34;search.exclude\u0026#34;: { \u0026#34;**/Intermediate/*\u0026#34;: true, \u0026#34;.vs/*\u0026#34;: true, \u0026#34;.cache/*\u0026#34;: true, \u0026#34;.vscode/*\u0026#34;: true, \u0026#34;**/*.dll\u0026#34;: true, \u0026#34;**/*.ilk\u0026#34;: true, \u0026#34;**/*.pdb\u0026#34;: true, \u0026#34;**/*.exe\u0026#34;: true, \u0026#34;**/*.uasset\u0026#34;: true }, \u0026#34;files.watcherExclude\u0026#34;: { \u0026#34;**/Intermediate/*\u0026#34;: true, \u0026#34;.vs/*\u0026#34;: true, \u0026#34;.cache/*\u0026#34;: true, \u0026#34;.vscode/*\u0026#34;: true, \u0026#34;**/*.dll\u0026#34;: true, \u0026#34;**/*.ilk\u0026#34;: true, \u0026#34;**/*.pdb\u0026#34;: true, \u0026#34;**/*.exe\u0026#34;: true, \u0026#34;**/*.uasset\u0026#34;: true }, \u0026#34;clangd.arguments\u0026#34;: [ \u0026#34;-j=16\u0026#34;, \u0026#34;--pch-storage=memory\u0026#34;, \u0026#34;--clang-tidy\u0026#34;, \u0026#34;--rename-file-limit=0\u0026#34;, \u0026#34;--background-index\u0026#34; ] } } Note that we\u0026rsquo;re specifying that indenting should be done with tabs, and that they should be 4 spaces wide. This because this setup will follow Epic\u0026rsquo;s coding standard as closely as possible, which uses that. Other than the indentation, you can see I excluded a bunch of filetypes from showing up on search and from being watched, although I didn\u0026rsquo;t exclude them from the explorer view (the files.exclude entry is for that), since I want to be able to see all files there at all times.\nIgnore the \u0026ldquo;clangd\u0026rdquo; bit for now, we\u0026rsquo;ll get there!\nVSCode tasks VSCode let\u0026rsquo;s you setup tasks for automating any commands. It\u0026rsquo;s slightly better than keeping a bunch of batch scripts on-hand, since you can easily specify dependencies between them, have them show up on VSCode\u0026rsquo;s UI and can easily bind keyboad shortcuts. You can see the official docs for more details, but the TL;DR way to set them up is to make a \u0026ldquo;.vscode\u0026rdquo; folder on your workspace root (so next to the Engine folder), and to put a \u0026ldquo;tasks.json\u0026rdquo; file in there.\nWe\u0026rsquo;re going to be adding a lot of tasks here for everything you may want to do, but for now a valid \u0026ldquo;tasks.json\u0026rdquo; just looks like the below:\n{ // See https://go.microsoft.com/fwlink/?LinkId=733558 // for the documentation about the tasks.json format \u0026#34;version\u0026#34;: \u0026#34;2.0.0\u0026#34;, \u0026#34;tasks\u0026#34;: [ ] } Setting up clangd Unreal uses a custom build system with a lot of code generation, and the source files expect the include paths to be a certain way (relative to the modules), so overall I couldn\u0026rsquo;t get VSCode Intellisense to work with it. Clangd does actually work though, so here we\u0026rsquo;ll set it up for UE.\nDownload and install the latest stable release of LLVM. I\u0026rsquo;m using version 16.0.0. It should come with clangd bundled in (open a terminal and type clangd --version after the install to check), but you can install it directly here if that\u0026rsquo;s not the case for some reason.\nYou will now need to install the clangd extension for VSCode as well, so that it can use your clangd install for syntax highlighting, formatting and etc.\nClangd uses clang (a C++ compiler) and compiles your code under the hood, so in order to get clangd working on the UE source code and your project, you need to generate a file that tells clang how to compile everything. Luckily Unreal Build Tool (UBT) (which is UE\u0026rsquo;s build system) can do that! We\u0026rsquo;re going to setup a few VSCode tasks for that, so our \u0026ldquo;tasks.json\u0026rdquo; will look like this now:\n{ // See https://go.microsoft.com/fwlink/?LinkId=733558 // for the documentation about the tasks.json format \u0026#34;version\u0026#34;: \u0026#34;2.0.0\u0026#34;, \u0026#34;tasks\u0026#34;: [ { \u0026#34;label\u0026#34;: \u0026#34;Build UAT\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Builds the Unreal Automation Tool (which also builds UBT). Most other commands require this be built beforehand\u0026#34;, \u0026#34;type\u0026#34;: \u0026#34;shell\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Build/BatchFiles/BuildUAT.bat\u0026#34;, \u0026#34;group\u0026#34;: \u0026#34;build\u0026#34;, \u0026#34;presentation\u0026#34;: { \u0026#34;reveal\u0026#34;: \u0026#34;always\u0026#34;, \u0026#34;showReuseMessage\u0026#34;: false }, \u0026#34;promptOnClose\u0026#34;: false, \u0026#34;problemMatcher\u0026#34;: \u0026#34;$msCompile\u0026#34;, }, { \u0026#34;label\u0026#34;: \u0026#34;Regenerate compile_commands.json for the Unreal Editor\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Regenerates compile_commands.json and the .rsp.gcd files used by clangd\u0026#34;, \u0026#34;type\u0026#34;: \u0026#34;shell\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/DotNET/AutomationTool/UnrealBuildTool.exe\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-mode=GenerateClangDatabase\u0026#34;, \u0026#34;UnrealEditor\u0026#34;, \u0026#34;Development\u0026#34;, \u0026#34;Win64\u0026#34; ], \u0026#34;group\u0026#34;: \u0026#34;build\u0026#34;, \u0026#34;presentation\u0026#34;: { \u0026#34;reveal\u0026#34;: \u0026#34;always\u0026#34;, \u0026#34;showReuseMessage\u0026#34;: false }, \u0026#34;promptOnClose\u0026#34;: false, \u0026#34;problemMatcher\u0026#34;: \u0026#34;$msCompile\u0026#34;, \u0026#34;dependsOrder\u0026#34;: \u0026#34;sequence\u0026#34;, \u0026#34;dependsOn\u0026#34;: [\u0026#34;Build UAT\u0026#34;] }, { \u0026#34;label\u0026#34;: \u0026#34;Regenerate compile_commands.json for Project\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Regenerates compile_commands.json and the .rsp.gcd files used by clangd\u0026#34;, \u0026#34;type\u0026#34;: \u0026#34;shell\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/DotNET/AutomationTool/UnrealBuildTool.exe\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-mode=GenerateClangDatabase\u0026#34;, \u0026#34;-Project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;-OutputDir=\\\u0026#34;F:/ProjectName/\\\u0026#34;\u0026#34;, \u0026#34;-Filter=\\\u0026#34;.../ProjectName/Source/...\\\u0026#34;\u0026#34;, \u0026#34;ProjectNameEditor\u0026#34;, \u0026#34;Development\u0026#34;, \u0026#34;Win64\u0026#34; ], \u0026#34;group\u0026#34;: \u0026#34;build\u0026#34;, \u0026#34;presentation\u0026#34;: { \u0026#34;reveal\u0026#34;: \u0026#34;always\u0026#34;, \u0026#34;showReuseMessage\u0026#34;: false }, \u0026#34;promptOnClose\u0026#34;: false, \u0026#34;problemMatcher\u0026#34;: \u0026#34;$msCompile\u0026#34;, \u0026#34;dependsOrder\u0026#34;: \u0026#34;sequence\u0026#34;, \u0026#34;dependsOn\u0026#34;: [\u0026#34;Build UAT\u0026#34;] }, ] } After setting that up, you should be able to see your task and run the \u0026ldquo;Regenerate compile_commands.json for the Unreal Editor\u0026rdquo; task it by pressing Ctrl+Shift+B and picking it from the menu. After a little while (up to a couple minutes on my machine) it will pop a \u0026ldquo;compile_commands.json\u0026rdquo; on the workspace root, which is what we need (and right where we need it, clangd will search for it on the workspace root too). It will also create a bunch of \u0026ldquo;.rsp.gcd\u0026rdquo; files throughout the \u0026ldquo;Intermediate\u0026rdquo; folders of your target, that the \u0026ldquo;compile_commands.json\u0026rdquo; file includes.\nIt\u0026rsquo;s important to note a few things, however:\nIf you add a new source file to the Unreal Editor source, you need to run the task to describe how to compile it within \u0026ldquo;compile_commands.json\u0026rdquo;, otherwise clangd will not work. This is analogous to the recommendation that you should run the \u0026ldquo;GenerateProjectFiles.bat\u0026rdquo; to update the VS solution; When those tasks are run, they will create and overwrite the \u0026ldquo;.rsp.gcd\u0026rdquo; files. UBT will pick up on this and want to rebuild/relink those objects, and so the next time you build you\u0026rsquo;ll find your editor wants to recompile everything; If you delete your \u0026ldquo;Intermediate\u0026rdquo; folder to try \u0026ldquo;resetting things\u0026rdquo; for some reason, you will need to run this task again to recreate the \u0026ldquo;.rsp.gcd\u0026rdquo; files, or else clangd will not know anything about the types on the modules you modified; If you want to regenerate \u0026ldquo;compile_commands.json\u0026rdquo; for a subset of files, you can use the \u0026ldquo;-Filter\u0026rdquo; argument, which the third task above uses: You should run that task if you add any file to your project (here at \u0026ldquo;F:/ProjectName/ProjectName.uproject\u0026rdquo;). It will only touch that particular project\u0026rsquo;s source files, so it won\u0026rsquo;t recompile everything. There are two more configuration bits you will need. First of all, place a \u0026ldquo;.clangd\u0026rdquo; file on your workspace and put this in it:\nCompileFlags: Add: [-D__INTELLISENSE__, -ferror-limit=0] Diagnostics: UnusedIncludes: Strict These are how you pass additional definitions and arguments to clang while clangd is using it to compile your project internally. They come mostly from this great forum post where user drcxd suggests these, as the #ifdefs around the UCLASS macro only really expand to something Intellisense/clangd can reason about in case __INTELLISENSE__ is defined. As you can imagine, Intellisense defines that automatically while clangd does not, so we do that there.\nThe second bit of configuration left is to setup some settings for VSCode\u0026rsquo;s clangd extension. There are many settings, but I think important ones are the below:\n\u0026#34;clangd.arguments\u0026#34;: [ \u0026#34;-j=16\u0026#34;, \u0026#34;--pch-storage=memory\u0026#34;, \u0026#34;--clang-tidy\u0026#34;, \u0026#34;--background-index\u0026#34; ] We added this to our workspace settings already! This will have it build the background index and use up to 16 threads if it can. Keeping the PCH storage in memory supposedly helps with performance, and --clang-tidy is great for more analysis, although I believe it is enabled by default.\nAfter you set this up clangd will start building the background index. On my mid-range PC for 2023, while it did run in the background it took between 6 and a 12 hours to complete while pinning my PC to 100% CPU usage, which was sort of crazy, although I haven\u0026rsquo;t needed to do that again. This background index will live on the \u0026ldquo;.cache/\u0026rdquo; folder on the workspace root (which is why I excluded that folder on the workspace settings). It is possible that you do not need a background index and can just index on-demand, but I haven\u0026rsquo;t compared the performance yet.\nOther tasks Building, packaging, cooking and running UE are all done by directly interacting with UBT, and can be easily done via the command line (and so via VSCode tasks!). Here are a couple of useful things you may want to do. Note that these should just be additional entries on the tasks array in the \u0026ldquo;tasks.json\u0026rdquo;. For clarity I\u0026rsquo;ll omit the non-interesting part of the settings like group, type or problemMatcher, although keep in mind there are a bunch in there.\nBuilding Here are some basic tasks to build and lauch the editor by itself or with a project:\n{ \u0026#34;label\u0026#34;: \u0026#34;Build Unreal Editor\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Just builds the Unreal Editor without specifying a project\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Build/BatchFiles/Build.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-Target=\\\u0026#34;UnrealEditor Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Target=\\\u0026#34;ShaderCompileWorker Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Quiet\u0026#34;, \u0026#34;-WaitMutex\u0026#34;, \u0026#34;-FromMsBuild\u0026#34; ], }, { \u0026#34;label\u0026#34;: \u0026#34;Launch Unreal Editor\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Launch the Unreal Editor without a project\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/Win64/UnrealEditor.exe\u0026#34;, }, { \u0026#34;label\u0026#34;: \u0026#34;Build Project\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Actually builds the project\u0026#39;s Editor target\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Build/BatchFiles/Build.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-Project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;-Target=\\\u0026#34;ProjectNameEditor Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Target=\\\u0026#34;ShaderCompileWorker Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Quiet\u0026#34;, \u0026#34;-WaitMutex\u0026#34;, \u0026#34;-FromMsBuild\u0026#34; ], }, { \u0026#34;label\u0026#34;: \u0026#34;Launch Project\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Launch the Unreal Editor for the provided project\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/Win64/UnrealEditor.exe\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-Project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, ], }, Updating .generated.h files One of the first steps of the UE compilation process is that the Unreal Header Tool (UHT) parses your header files, like \u0026ldquo;file.h\u0026rdquo;. If they have any of the UE macros in them (like UCLASS, USTRUCT, UPROPERTY and etc.), UHT will create a \u0026ldquo;file.generated.h\u0026rdquo; file with a ton of additional code, which clangd does need to read to provide sensible feedback.\nSometimes when these header files (like \u0026ldquo;file.h\u0026rdquo;) are modified (say by adding new UPROPERTYs or modifying them), the contents of that file and \u0026ldquo;file.generated.h\u0026rdquo; can start to diverge, and clangd can start to spew some nonsense. You could run the full \u0026ldquo;Build Project\u0026rdquo; task for it, but a trick is that you can run it with the \u0026ldquo;-SkipBuild\u0026rdquo; argument to just have it update the \u0026ldquo;.generated.h\u0026rdquo; files instead. This only takes a few seconds, and can run in the background. Some people set this up to run every 5 minutes or after every save, but I just use a separate task for it:\n{ \u0026#34;label\u0026#34;: \u0026#34;Regenerate headers\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Regenerate the .generated.h files in case you modify any headers\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Build/BatchFiles/Build.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-Target=\\\u0026#34;ProjectNameEditor Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Target=\\\u0026#34;ShaderCompileWorker Win64 Development\\\u0026#34;\u0026#34;, \u0026#34;-Project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;-SkipBuild\u0026#34; ], }, Generating the Visual Studio solution If for some reason you want to debug with VS instead of VSCode (we\u0026rsquo;ll get to the debugging part later in the post though) you can use the following task to just regenerate the VS solution file anyway, launch it and debug with it. I wanted to add this because there are some potentially useful arguments you may not know about, which make this generation step slightly faster, and also prevent it from generating useless projects on the solution file.\n{ \u0026#34;label\u0026#34;: \u0026#34;Generate project files\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Generate the Visual Studio solution file in case you want to use it for debugging\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/GenerateProjectFiles.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;ProjectName.uproject\u0026#34;, // Just the project for this one target \u0026#34;-Game\u0026#34;, \u0026#34;-NoIntellisense\u0026#34;, // If we use VS, it will be just for debugging anyway \u0026#34;-NoShippingConfigs\u0026#34;, \u0026#34;-CurrentPlatform\u0026#34;, // Generates configurations for current platform only \u0026#34;-NoDotNet\u0026#34; // Prevents the generation of UnrealBuildTool and other C# projects ], }, Full update One thing that was useful for me was to setup a \u0026ldquo;full resync\u0026rdquo; task that I can run automatically at the end of the day/week to get UE ready for the next day. You can do that with something like this:\n{ \u0026#34;label\u0026#34;: \u0026#34;Full resync\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Full update: Resyncs source control and builds everything\u0026#34;, \u0026#34;dependsOrder\u0026#34;: \u0026#34;sequence\u0026#34;, \u0026#34;dependsOn\u0026#34;: [ \u0026#34;Resync source control\u0026#34;, \u0026#34;Regenerate compile_commands.json for the Unreal Editor\u0026#34;, \u0026#34;Regenerate compile_commands.json for Project\u0026#34;, \u0026#34;Build Project\u0026#34;, \u0026#34;Launch Project\u0026#34; ] } I omitted the \u0026ldquo;Resync source control\u0026rdquo; for now, but you get the picture.\nPrecompiling shaders Note how I call the \u0026ldquo;Launch project\u0026rdquo; task at the end there. The intent is to get it to precompile the shaders that it will need to open the editor. With UE 5 this will only compile the shaders needed to open the editor though (i.e. assets used on your startup level). You may want a more shotgun approach and to have it compile all shaders and build all static meshes and so on. Apparently Epic does something similar on a nightly basis. You can use the following task for that, although keep in mind when I did this for a medium size project it took about 10 hours. I tried to restrict it only to the WindowsEditor platform, but to be perfectly honest I\u0026rsquo;m not sure if that has an effect or not.\n{ \u0026#34;label\u0026#34;: \u0026#34;Fill derived data\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Compile shaders, builds meshes and does all the required one-time setup for your project so the editor opens fast\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/Win64/UnrealEditor-Cmd.exe\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;ProjectName\u0026#34;, \u0026#34;-run=DerivedDataCache\u0026#34;, \u0026#34;-targetplatform=WindowsEditor\u0026#34;, \u0026#34;-fill\u0026#34; ], }, Packaging and cooking Finally, here is how you\u0026rsquo;d cook or package your project (packaging involves cooking):\n{ \u0026#34;label\u0026#34;: \u0026#34;Package project\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Cooks and fully packages the project for target platform and configuration\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/RunUAT.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-ScriptsForProject=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;BuildCookRun\u0026#34;, \u0026#34;-project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;-target=ProjectName\u0026#34;, \u0026#34;-platform=Win64\u0026#34;, \u0026#34;-clientconfig=Development\u0026#34;, \u0026#34;-utf8output\u0026#34;, \u0026#34;-nocompileeditor\u0026#34;, \u0026#34;-skipbuildeditor\u0026#34;, \u0026#34;-build\u0026#34;, \u0026#34;-cook\u0026#34;, \u0026#34;-stage\u0026#34;, \u0026#34;-pak\u0026#34;, \u0026#34;-archive\u0026#34;, \u0026#34;-archivedirectory=\\\u0026#34;C:/Output/Directory\\\u0026#34;\u0026#34; ], \u0026#34;dependsOrder\u0026#34;: \u0026#34;sequence\u0026#34;, \u0026#34;dependsOn\u0026#34;: [\u0026#34;Build UAT\u0026#34;, \u0026#34;Build Project\u0026#34;] }, { \u0026#34;label\u0026#34;: \u0026#34;Cook content\u0026#34;, \u0026#34;detail\u0026#34;: \u0026#34;Cook content for a target platform and configuration\u0026#34;, \u0026#34;command\u0026#34;: \u0026#34;${workspaceFolder}/RunUAT.bat\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-ScriptsForProject=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;BuildCookRun\u0026#34;, \u0026#34;-project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34;, \u0026#34;-target=ProjectName\u0026#34;, \u0026#34;-platform=Win64\u0026#34;, \u0026#34;-clientconfig=Development\u0026#34;, \u0026#34;-utf8output\u0026#34;, \u0026#34;-nocompileeditor\u0026#34;, \u0026#34;-skipbuildeditor\u0026#34;, \u0026#34;-cook\u0026#34;, ], \u0026#34;dependsOrder\u0026#34;: \u0026#34;sequence\u0026#34;, \u0026#34;dependsOn\u0026#34;: [\u0026#34;Build UAT\u0026#34;, \u0026#34;Build Project\u0026#34;] }, These tasks are easy to construct on your own, or update to whatever arguments you use. Just launch the packaging/cooking task from the editor and keep an eye out on the Output Log: One of the first lines emitted will be the command and all required arguments to run your exact packaging/cooking task, which you can then extract and put it on a VSCode task.\nThe only thing I may add here is that I recommend having explicit dependencies on the UAT and \u0026ldquo;Build Project\u0026rdquo; tasks, as personally sometimes I forget to do this and it can lead to bizarre errors when trying to package.\nClang-format This is a great and very configurable code formatter for C++ (and a bunch of other languages) that also uses clang. There is an extension for VSCode you can install.\nYou can configure it with some pre-existing styles like Google or WebKit, or setup a custom style, which is what we\u0026rsquo;ll do. For a custom style all you need to do is place a \u0026ldquo;.clang-format\u0026rdquo; file on your workspace root (and within each UE project\u0026rsquo;s root!) with your chosen settings. That is usually enough for the extension to find it and use it, but there are some settings on the extension to have it point at a specific file path if you want.\nI went over Epic\u0026rsquo;s coding standard and all the different clang-format options, and found the values that best match. The standard itself is not very strict with the styling, but UE code has a distinct \u0026ldquo;look\u0026rdquo;, which I tried to approach as much as possible. I used some references though, like this one, this other one but especially this one, that has an include category trick to put the generated files in the right position.\nHere it is: My .clang-format\nDebugging It would be perfectly viable to debug using VS (by using the \u0026ldquo;Generate project files\u0026rdquo; task above and disabling Intellisense it\u0026rsquo;s not that bad). You can keep it open in the background and just click the green play button to launch every time, but I really didn\u0026rsquo;t want to have to keep two different editors open at all times.\nLuckily the debugging workflow for C++ in VSCode is not so bad! And it has all the features I need, like data breakpoints and being able to easily see the callstacks of different threads. It\u0026rsquo;s pretty easy to set it up: All you need is to place a \u0026ldquo;launch.json\u0026rdquo; file within the \u0026ldquo;.vscode\u0026rdquo; folder (so next to the \u0026ldquo;tasks.json\u0026rdquo; file we used earlier). Here is the official documentation about it. There are many other options like making configurations that attach to existing processes instead of launching them, and so on.\nMy full \u0026ldquo;launch.json\u0026rdquo; looks like this:\n{ \u0026#34;configurations\u0026#34;: [ { \u0026#34;name\u0026#34;: \u0026#34;Build and debug\u0026#34;, \u0026#34;type\u0026#34;: \u0026#34;cppvsdbg\u0026#34;, \u0026#34;request\u0026#34;: \u0026#34;launch\u0026#34;, \u0026#34;cwd\u0026#34;: \u0026#34;${workspaceRoot}\u0026#34;, \u0026#34;program\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Binaries/Win64/UnrealEditor.exe\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;-project=\\\u0026#34;F:/ProjectName/ProjectName.uproject\\\u0026#34;\u0026#34; ], \u0026#34;internalConsoleOptions\u0026#34;: \u0026#34;openOnSessionStart\u0026#34;, \u0026#34;visualizerFile\u0026#34;: \u0026#34;${workspaceFolder}/Engine/Extras/VisualStudioDebugging/Unreal.natvis\u0026#34;, \u0026#34;console\u0026#34;: \u0026#34;internalConsole\u0026#34;, \u0026#34;preLaunchTask\u0026#34;: \u0026#34;Build Project\u0026#34; } ], \u0026#34;version\u0026#34;: \u0026#34;2.0.0\u0026#34; } Note the visualizerFile entry: It points at the natvis file that ships with the engine, which is required in order to be able to inspect the UE types like FString in a usable manner. Luckily VSCode has a full implementation of the natvis handling stuff!\nAlso note that you can have a preLaunchTask, where you specify the label value for the task you want to run before launching the program. Unfortunately you can\u0026rsquo;t easily pass any arguments or data between the launch configuration and the task\u0026hellip; There are many issues on VSCode\u0026rsquo;s GitHub page about this, but at the time of writing this is not yet supported. This means you\u0026rsquo;ll need a separate launch/task pair if you want to test different platforms/configurations/project names/targets.\nOne other thing is that there is no easy way of using this launch configuration to \u0026ldquo;launch without debugging\u0026rdquo; for some reason, which is why I setup those dedicated tasks to do that instead!\nAnyway, after this is setup you can switch VSCode to Run mode with Ctrl+Shift+D (or by clicking on View -\u0026gt; Run), and the \u0026ldquo;Build and debug\u0026rdquo; will show up as a configuration you can launch. Breakpoints, inspecting variables, stepping through callstacks and threads should work exactly the same as in VS.\nUE\u0026rsquo;s default VSCode editor setup Now that you have a full grasp of what I went for, you may want to know that it\u0026rsquo;s possible to run the \u0026ldquo;GenerateProjectFiles.bat\u0026rdquo; file on the UE install with the \u0026ldquo;-VSCode\u0026rdquo; argument (you also get the same effect if you set VSCode as your source code editor on your UE Editor settings and then just run GenerateProjectFiles.bat with no arguments). This will automatically generate some \u0026ldquo;tasks.json\u0026rdquo; and \u0026ldquo;launch.json\u0026rdquo; files for your targets, which you may want to check out. Keep in mind that when you do this it will overwrite your existing files, so save your old \u0026ldquo;tasks.json\u0026rdquo; and \u0026ldquo;launch.json\u0026rdquo;!\nI really disliked the output tasks and launch configurations from that command though, as it generates hundreds of different items for targets/configurations I don\u0026rsquo;t really need, and doesn\u0026rsquo;t generate the tasks I do actually want!\nSetting VSCode as the source code editor on the UE Editor settings sadly also doesn\u0026rsquo;t mean it can open VSCode whenever you want to e.g. see the source of a Blueprint node or open your project. It just doesn\u0026rsquo;t really do anything for me at all in those situations.\nStep-by-step setup Now that we know all the aspects involved, here is what you\u0026rsquo;d actually do if you want to set this up for yourself locally. I\u0026rsquo;m assuming you are building UE from the Github source, and don\u0026rsquo;t have a UE project setup yet.\nClone the Github source git clone --single-branch --branch 5.2 https://github.com/EpicGames/UnrealEngine.git; Run the contained \u0026ldquo;Setup.bat\u0026rdquo; to download the rest of the dependencies; Grab the files we described throughout this post from my Github repo or copy the desired snippets from this post; Open \u0026ldquo;Engine\\Source\\Runtime\\Core\\Public\\GenericPlatform\\GenericPlatformProcess.h\u0026rdquo; and change the line 816 (as of UE 5.2) from this: #\terror Unsupported architecture! Into this:\n# ifndef __INTELLISENSE__ #\terror Unsupported architecture! # endif If you don\u0026rsquo;t do this, this \u0026ldquo;Unsupported architecture!\u0026rdquo; error will show up on almost every file\u0026hellip; This change just prevents the error from showing up as a problem for clangd, but it won\u0026rsquo;t affect anything else. We don\u0026rsquo;t really care about the missing \u0026ldquo;pause\u0026rdquo; intrinsics for clangd\u0026rsquo;s build as it won\u0026rsquo;t actually be run anyway!\nRun the \u0026ldquo;Build Unreal Editor\u0026rdquo; target (by pressing Ctrl+Shift+B in VSCode and picking from the list); Run the \u0026ldquo;Launch Unreal Editor\u0026rdquo; target to open the UE project picker and create a new project; Add your new project as a folder to your VSCode workspace if you want (by right-clicking open space on the exporer view in VSCode); Rename the \u0026ldquo;ProjectName\u0026rdquo; instances to your actual project name and path within \u0026ldquo;tasks.json\u0026rdquo; and \u0026ldquo;launch.json\u0026rdquo;; Run the \u0026ldquo;Regenerate compile_commands.json for the Unreal Editor\u0026rdquo; task; Run the \u0026ldquo;Regenerate compile_commands.json for Project\u0026rdquo; task; Run the \u0026ldquo;Build Project\u0026rdquo; task. That\u0026rsquo;s it! Everything should be working now. From now you can run the \u0026ldquo;Launch Project\u0026rdquo; task to just launch it without debugging, or launch the debugging configuration we talked about on the Debugging section.\nHere\u0026rsquo;s how the rest of the workflow works:\nYou can modify files on the project source freely, and just hit Recompile (or enable Live Coding) in the editor to hot reload the changes; Any time you modify a header file too much, you need to either rebuild or run the \u0026ldquo;Regenerate headers\u0026rdquo; task, or clangd will start generating garbage; Any time you add a new source file to your project, you may need to run the \u0026ldquo;Regenerate compile_commands.json for Project\u0026rdquo; task (sometimes clangd can go a long time without needing this, somehow); Any time you add a new source file to the UE source, you may need to run the \u0026ldquo;Regenerate compile_commands.json for Unreal Editor\u0026rdquo; task. Comparisons with VS After using it for a few weeks, I can safely say this is a superior workflow (for me!). I don\u0026rsquo;t think I\u0026rsquo;ll be going back any time soon, but here are some bullet points:\nThe good:\nVSCode opens instantly and feels overall much snappier to use than VS (low bar though); Clangd with clang-tidy is great at spotting things like unused variables or includes, and you can e.g. set up your \u0026ldquo;.clangd\u0026rdquo; file to use something like \u0026ldquo;-Wall\u0026rdquo; and other flags to get as much feedback as you want; It works great with other VSCode extensions like \u0026ldquo;Error lens\u0026rdquo;, which can show the error messages directly inline with your code; Opening a file shows the basic regex-based syntax highlighting instantly, and after clangd finishes loading the index for a file (or building it) it shows the proper, AST-based highlighing, which seems to stay loaded in memory for the whole session. On VS even after giving VS Intellisense 32GB of RAM to work with, after switching through a handful of files it will drop the index for a previous file. Opening an unindexed file on VS Intellisense will just show absolutely no syntax highlighing of any kind; \u0026ldquo;Go to definition\u0026rdquo; works really well, and it\u0026rsquo;s quite interesting to do it on a macro and see where it goes. I work with third-party libraries as well, and it can index into them no problem; Having all of these custom tasks setup and a couple keystrokes away is great. I work with multiple projects at a time so it\u0026rsquo;s great to be working on one and start packaging this other project instantly without having to change \u0026ldquo;solution files\u0026rdquo; or anything of the sort. I like that the clangd error messages come from clang, and if you build on Windows you\u0026rsquo;ll usually end up using MSVC. This means you\u0026rsquo;re checking your code with two compilers, so that\u0026rsquo;s a higher chance of catching errors. The bad:\nNot exactly straightforward (look how long this blog post is, for starters); For some reason it\u0026rsquo;s sometimes bad at finding the header/cpp file pairs? You\u0026rsquo;ll have file.h and file.cpp open and indexed, and doing the shortcut to switching between Source/Header will send me to \u0026ldquo;ObjectMacros.h\u0026rdquo; or somewhere bizarre? Building the clangd background index takes a ridiculous amount of time. I don\u0026rsquo;t have any comparisons with the performance with it enabled or disabled though, so maybe that\u0026rsquo;s worth investigating; When right-clicking a Blueprint node and picking to show the C++ implementation, UE will default to opening the VS solution file (which doesn\u0026rsquo;t exist) and show an error toast instead. I haven\u0026rsquo;t tried configuring this yet though; Searching for files by name takes a bit longer than VS for some reason (although it does seem to cache these afterwards). I\u0026rsquo;ve tried restricting the file search on the workspace settings but it\u0026rsquo;s still slow; I\u0026rsquo;ve noticed that the debugger takes a while longer to load local variables sometimes, and so stepping through code can be slower than in VS (believe it or not); The VSCode debugger interface doesn\u0026rsquo;t have a \u0026ldquo;run program up to this line\u0026rdquo; feature. I remember laughing at this feature when VS released it (as you can achieve the same by setting a new breakpoint, hiting Continue, and then removing it) but I actually miss it; Once every couple of days clangd shows some incorrect analysis for a file, like it failed to parse something, and it won\u0026rsquo;t go away until I close the file and open it again; It takes a comparable amount of time to parse a file compared to Intellisense for me. It\u0026rsquo;s not bad per se, just disappointing. This is not so bad as it seems to retain that in memory for way longer than VS Intellisense however; Conclusions and next steps If you scrolled down here and missed it, here is my Github repo with everything described in this blog post. You can mostly copy-and-paste those files onto an UE (from Github) installation and be good to go.\nOverall, if you prefer VSCode I\u0026rsquo;d say this approach is worth it. It doesn\u0026rsquo;t take that long to setup: This article is really just this long because I\u0026rsquo;m very bad at summarizing.\nThere are many things I want to improve with this, for example:\nMake a more seamless \u0026ldquo;first setup\u0026rdquo; task/script to setp all this up; Somehow find a way of not having to spell out the UE project\u0026rsquo;s path and target for every single task\u0026hellip; there are ways using environment variables/env files/other extensions, but they all seemed to just add too much more complication into the mix; Find a way of having UE properly call into the open VSCode workspace when it wants to show source code for whatever reason; I believe there must be ways of making the clangd parsing faster by tweaking it\u0026rsquo;s compilation on the \u0026ldquo;.clangd\u0026rdquo; file, although I haven\u0026rsquo;t tried that much just yet. But for now, this should be all I wanted to show about my current setup. Hopefully this helps you with yours!\nThanks for reading!\n","permalink":"https://1danielcoelho.github.io/unreal-engine-vscode-workflow/","summary":"\u003cp\u003eA good part of my day consits of waiting for Visual Studio (VS) to respond. I work with Unreal Engine (UE) on Windows and VS seems to be the recommended IDE for it, even though it\u0026rsquo;s not really required by UE or its build system in any way. There are many other IDE choices out there, but I tend to use VSCode for other projects and note-taking, so I wanted to try exclusively using it for UE as well, and this is what I got.\u003c/p\u003e","title":"Unreal Engine development workflow using just VSCode"},{"content":"I\u0026rsquo;ve been getting into machine learning with Pytorch these past few months, and one of my notes which has gotten the most mileage is this \u0026ldquo;deconfuser\u0026rdquo; note where I write out all the useful conventions that I need, but occasionally forget. I figured there\u0026rsquo;s a chance somebody finds them useful, and it would be a good simple post to get some traction and get back to blogging more.\nHere it is:\nConventions Imagine a neural network with a single 3x4 linear layer:\nNote that it has two columns of circles (\u0026ldquo;neurons\u0026rdquo;), but it is a single layer: Its easier to think of the layer as the thing between the neurons instead of an actual layer of neurons (which would be one of those columns).\nYou can describe the layer\u0026rsquo;s weights and biases with matrices W and B like this:\nNotice their shapes on the bottom right (number of rows x number of columns). Sometimes you will see W drawn transposed instead, like how I\u0026rsquo;ve also drawn it on the right in the above image (in this case with 3 rows and 4 columns).\nThis is how you describe that layer in Pytorch:\nimport torch layer = torch.nn.Linear(in_features=3, out_features=4, bias=True) print(layer.weight.shape) # Prints [4, 3] print(layer.bias.shape) # Prints [4] Note that it has a 4x3 matrix of weights, which is why I chose to draw W in that way. Also note that technically the bias is a line vector (it has only one dimension with 4 values) instead of being a 1x4 row vector. It\u0026rsquo;s more useful to think of these as 1xN row vectors instead of line vectors though, for reasons we\u0026rsquo;ll get to at the Broadcasting section below.\nThe layer receives a 3-dimensional input X, and produces a 4-dimensional output Y. Again, they\u0026rsquo;re drawn vertically (kinda like column vectors) on the neural network diagram because it sort of fits, but in Pytorch we\u0026rsquo;d rather think of them as row vectors.\nThis is how you apply that layer to a tensor X and receive a tensor Y in Pytorch:\nX = torch.rand(3) layer = torch.nn.Linear(in_features=3, out_features=4, bias=True) # The generic way of applying any layer Y = layer(X) # What torch.nn.Linear does internally Y_manual = X @ layer.weight.transpose(0, 1) + layer.bias assert (Y == Y_manual).all().item() # y and y_manual are the exact same The @ operator actually just [maps][https://github.com/pytorch/pytorch/blob/6dcc214ac273d594b8a9a30e1f90e30a3c1e40c8/torch/_tensor.py#L885] to [torch.matmul][https://pytorch.org/docs/stable/generated/torch.matmul.html], which is a regular matrix multiply.\nThe .transpose(0, 1) just transposes the dimensions 0 and 1, so rows with columns. Drawing it out, Y and Y_manual are both computed by the simple linear layer function:\nAccording to the matrix multiply rules, you\u0026rsquo;ll see you can perform the multiplication of matrices with shapes [1, 3] * [3, 4], (as the two numbers closer together are the same (3)) and that it will lead to a result with shape [1, 4] (the two numbers that are further apart), so that everything works out. Well, except for how X is not really a 1x3 row vector and is really a line vector\u0026hellip; I think it\u0026rsquo;s time:\nBroadcasting I\u0026rsquo;ve been saying that X, Y and B are row vectors so far, but in Pytorch they\u0026rsquo;re 1-dimensional line vectors. Who\u0026rsquo;s to say they don\u0026rsquo;t represent a column instead of a row?\nWell Pytorch has this mechanism called \u0026ldquo;broadcasting\u0026rdquo;, where tensors can receive extra dimensions to make their shapes match up with other tensor shapes, in order to be able to perform some operation on them.\nFor example, look at this:\nt1 = torch.tensor([1, 2, 3]) # Shape [3] \u0026lt;--- line vector t2 = torch.tensor([[10, 20, 30], [40, 50, 60]]) # Shape [2, 3] \u0026lt;--- 2D matrix s = t1 + t2 print(s) # Prints [[11, 22, 33], [41, 52, 63]] print(s.shape) # Prints [2, 3] \u0026lt;--- 2D matrix Note that it \u0026ldquo;broadcast\u0026rdquo; (here meaning copy-pasted) the t1 tensor values into two identical rows of a \u0026ldquo;temporary tensor\u0026rdquo; [[1, 2, 3], [1, 2, 3]] that could then be added with t2 element-wise.\nTo know if the broadcast mechanism can help your case, you can do this rule of thumb:\nTensor shapes are aligned to the right: t2: [2, 3] t1: [3] Expand the tensor with fewer dimensions to have \u0026ldquo;1\u0026rdquo; for all the other dimensions: t2: [2, 3] t1: [1, 3] Pytorch will \u0026ldquo;broadcast\u0026rdquo; if and only if for each dimension you have an equal number of values (like how we have \u0026ldquo;3\u0026rdquo; for the right-most dimension there), or one of the tensors has just 1 value (like how we have 2 and 1 for the left-most dimension there). If one of the tensors has a single value for a dimension, Pytorch will just copy-paste that value until that the two tensors end up with the same number of values for that particular dimension (which is what we saw on the snippet above when we did s = t1 + t2) This is why I said 1-dimensional vectors are kind of the same as row-vectors: Pytorch will broadcast them from being [N] to [1, N] before you try performing an operation (like a matrix multiply).\nThis is why we could do this:\nlayer = torch.nn.Linear(in_features=3, out_features=4, bias=True) X = torch.rand(3) # Shape [3] Wt = layer.weight.transpose(0, 1) # Shape [3, 4] B = layer.bias # Shape [4] Y_manual = X @ Wt + B # Shapes: [3] @ [3, 4] + [4] # Shapes: [1, 3] @ [3, 4] + [4] (after broadcasting X) # Shapes: [1, 4] + [4] (after matrix multiply) # Shapes: [1, 4] + [1, 4] (after broadcasting B) # Shapes: [1, 4] (after adding B) Batch matrix multiply There\u0026rsquo;s one last thing I think still fits in this post: In Pytorch models you rarely have a single 2D matrix for a tensor: You\u0026rsquo;ll have many more dimensions (batch, channels, etc.). What actually happens if we do something like this?\nX = torch.rand((10, 2, 3)) # Shape [10, 2, 3] # Like before, this has a [3, 4] weight tensor and a [4] bias tensor layer = torch.nn.Linear(in_features=3, out_features=4, bias=True) # These are all identical, and end up with shape [10, 2, 4] Y = layer(X) Y_manual = X @ layer.weight.transpose(0, 1) + layer.bias Y_matmul = torch.matmul(X, layer.weight.transpose(0, 1)) + layer.bias You can think of that 3-dimensional X as being 10 groups of [2, 3] matrices. When you apply layer (or use the @ operator, or call torch.matmul), Pytorch is going to broadcast layer\u0026rsquo;s weight and biases, and then perform each of the ten [2, 3] * [3, 4] matrix multiplies independently, returning a [10, 2, 4] tensor.\nThere is also a dedicated [torch.bmm][https://pytorch.org/docs/stable/generated/torch.bmm.html] function for performing the batch matrix multiply. However, annoyingly, this function doesn\u0026rsquo;t perform broadcasting and only works when the two arguments are tensors with exactly 3 dimensions:\nlayer = torch.nn.Linear(in_features=3, out_features=4, bias=True) X = torch.rand((20, 2, 3)) # Shape [20, 2, 3] Wt = layer.weight.transpose(0, 1) # Shape [3, 4] Wte = Wt.expand(20, 3, 4) # Shape [20, 3, 4] Y = torch.bmm(X, Wt) # Raises an error Y = torch.bmm(X, Wte) # OK Finally, what happens if you do a matmul with both tensor arguments having more than 2 dimensions?\nA = torch.tensor([ # Shape [2, 2, 3] [[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]] ]) B = torch.tensor([ # Shape [2, 3, 4] [[2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], [[2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], ]) C = A @ B # Shape [2, 2, 4] # C corresponds to torch.tensor([ # [[ 20, 20, 20, 20] # [ 47, 47, 47, 47]], # # [[ 74, 74, 74, 74], # [101, 101, 101, 101]] #]) assert (C[0] == A[0] @ B[0]).all().item() assert (C[1] == A[1] @ B[1]).all().item() Like the asserts suggest, it just does 2-dimensional matrix multiplies pairwise for the two arguments, regardless of how many higher dimensions they have. This means you must have exactly the same number of values in each of the higher dimensions though, and something like this wouldn\u0026rsquo;t work:\nA = torch.tensor([ # Shape [2, 2, 3] [[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]] ]) B_diff = torch.tensor([ # Shape [3, 3, 4] [[2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], [[2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], [[2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]], ]) D = A @ B_diff # Raises an error, since A and B_diff have incompatible shapes Broadcasting would also help you in this case though, so you could perform this batch matrix multiplication in these cases:\nA = torch.rand((2, 2, 3)) # Theses are the shapes of the tensors B = torch.rand((2, 3, 4)) C = A @ B # Ok A = torch.rand((2, 2, 3)) B = torch.rand( (3, 4)) C = A @ B # Ok A = torch.rand((2, 2, 3)) B = torch.rand((1, 3, 4)) C = A @ B # Ok A = torch.rand((1, 2, 2, 3)) B = torch.rand( (1, 3, 4)) C = A @ B # Ok A = torch.rand((1, 2, 2, 3)) B = torch.rand((4, 1, 3, 4)) C = A @ B # Ok A = torch.rand((2, 2, 2, 3)) B = torch.rand((4, 1, 3, 4)) C = A @ B # Error A = torch.rand((1, 2, 2, 3)) B = torch.rand( (4, 3, 4)) C = A @ B # Error A = torch.rand((1, 2, 2, 3)) B = torch.rand( (3, 4)) C = A @ B # Ok That’s about as much as I wanted to cover on this one. Some of these things always confused me, like how you specify a linear layer like torch.nn.Linear(3, 4) and it actually has a 4x3 weights matrix, but transposes it for the multiply.\nI deliberately chose different numbers of values in all dimensions, but if you have square matrices (more often than not) you can see how this can lead to subtle issues and confusion.\nThanks for reading!\n","permalink":"https://1danielcoelho.github.io/pytorch-conventions/","summary":"\u003cp\u003eI\u0026rsquo;ve been getting into machine learning with Pytorch these past few months, and one of my notes which has gotten the most mileage is this \u0026ldquo;deconfuser\u0026rdquo; note where I write out all the useful conventions that I need, but occasionally forget. I figured there\u0026rsquo;s a chance somebody finds them useful, and it would be a good simple post to get some traction and get back to blogging more.\u003c/p\u003e","title":"Pytorch and neural network conventions and notations"},{"content":"This post describes a simple and minimal workflow for developing Rust apps targetting WASM and running them on the browser with minimal iteration time.\nThe official guides for working with Rust and WebAssembly are great (Rust and WebAssembly, The wasm-bindgen Guide), but they really railroad you into a setup where on top of Cargo you need npm, wasm-pack and webpack. You haven\u0026rsquo;t even run anything yet and you have two package managers and two bundlers!\nDepending on what you want to do, you can achieve the same with a lot less. Lets go through the bare minimum first, and then we\u0026rsquo;ll make it slightly more ergonomic.\nNote that this is roughly what Ian Kettlewell described here. On that post the author mostly glanced over the setup and focused on his actual game though, but here we\u0026rsquo;ll focus on the workflow side.\nThe bare minimum For the bare minimum, you\u0026rsquo;ll need:\nwasm-bindgen; Run rustup target add wasm32-unknown-unknown once to make sure you can build to the wasm target; Your project structure should look like this:\ndist/ src/ lib.rs www/ index.html Cargo.toml build.bat Here\u0026rsquo;s what my Cargo.toml looks like:\n[package] name = \u0026#34;wasm_bindgen_test\u0026#34; version = \u0026#34;0.1.0\u0026#34; authors = [\u0026#34;Daniel Coelho\u0026#34;] edition = \u0026#34;2018\u0026#34; [lib] crate-type = [\u0026#34;cdylib\u0026#34;] [dependencies] wasm-bindgen = \u0026#34;0.2.80\u0026#34; Here\u0026rsquo;s lib.rs:\nuse wasm_bindgen::prelude::*; #[wasm_bindgen] pub fn add(a: i32, b: i32) -\u0026gt; i32 { return a + b; } And here\u0026rsquo;s what build.bat looks like:\n@echo off cargo build --target wasm32-unknown-unknown wasm-bindgen --out-dir dist --target web --no-typescript target\\wasm32-unknown-unknown\\debug\\wasm_bindgen_test.wasm echo f | xcopy /s /f /y www\\index.html dist\\index.html It should be pretty clear, but all that its doing is building to the wasm target, running wasm-bindgen to process the results and output to the dist folder, and copying the html file over there too.\nThe html file is roughly the default:\n\u0026lt;!DOCTYPE html\u0026gt; \u0026lt;html\u0026gt; \u0026lt;head\u0026gt; \u0026lt;meta content=\u0026#34;text/html;charset=utf-8\u0026#34; http-equiv=\u0026#34;Content-Type\u0026#34; /\u0026gt; \u0026lt;/head\u0026gt; \u0026lt;body\u0026gt; \u0026lt;script type=\u0026#34;module\u0026#34;\u0026gt; import init,{add} from \u0026#39;./wasm_bindgen_test.js\u0026#39;; async function run() { await init(); const result=add(1,2); console.log(`1 + 2 = ${result}`); if(result!==3) throw new Error(\u0026#34;wasm addition doesn\u0026#39;t work!\u0026#34;); } run(); \u0026lt;/script\u0026gt; \u0026lt;/body\u0026gt; \u0026lt;/html\u0026gt; That\u0026rsquo;s pretty much it. To build you just run build.bat, and it will do only what you actually need, and put everything on the dist folder. If your rust code is including functions exported from a Javascript file, wasm-bindgen will actually move that file to the dist folder on its own too, which is nice! (I don\u0026rsquo;t have this setup on this example for simplicity\u0026rsquo;s sake).\nAnnoyingly you can\u0026rsquo;t just double-click your html and look at the result on the browser, as it will prevent you from fetching the actual .wasm file due to CORS. There is likely a clever way around it, but realistically you\u0026rsquo;re going to be doing the stuff in the next section anyway, and then it ceases to be an issue.\nErgonomics upgrade To make this slightly handier, we\u0026rsquo;ll do what Ian suggested and setup a dev server and automatic reloading whenever we save a file.\nFor this part, you\u0026rsquo;ll need to setup two crates:\ncargo-watch (just run cargo install cargo-watch); devserver (just run cargo install devserver); There are other alternatives if all you want is a dev server with automatic reloading (you can run a Python command, just use some vscode extensions, etc.) but I like that its just another Rust crate instead.\nOnce the dependencies are set up, just make a new file on your project root named dev.bat, and put this in there:\nSTART \u0026#34;\u0026#34; devserver --path dist --reload cargo watch -d 0.05 -- build.bat It should be pretty obvious: It will host the files on the dist folder (by default on http://localhost:8080/ but you can change it). The second line starts cargo watch to observe your project and run build.bat 0.05 seconds after any file changes (the wait period is there as some file operations trigger multiple filesystem notices in quick succession). You can also specify glob patterns to ignore particular files or directories, but by default cargo watch will ignore anything on your .gitignore, which tends to work pretty well.\nAlso note that the first command is run with START to run it on a separate command prompt (as it will block it while it runs) that you can mostly put away. The second command will run on your current terminal, which is good because that\u0026rsquo;s where your cargo build output (compile errors and so on) will end up, which you likely want to keep an eye on.\nThat should be it! When you want to start working, just run dev.bat on your terminal and open http://localhost:8080/ on a browser. On my machine it takes about a second to go from hitting Ctrl+S to save some file and seeing the updated page (the first couple of reloads will be a bit slower than usual of course). Even on larger projects it has never surpassed a few seconds for me.\nLet me know if you know how to simplify or make this even faster!\n","permalink":"https://1danielcoelho.github.io/fast-rust-wasm-workflow/","summary":"\u003cp\u003eThis post describes a simple and minimal workflow for developing Rust apps targetting WASM and running them on the browser with minimal iteration time.\u003c/p\u003e\n\u003cp\u003eThe official guides for working with Rust and WebAssembly are great (\u003ca href=\"https://rustwasm.github.io/docs/book/\"\u003eRust and WebAssembly\u003c/a\u003e, \u003ca href=\"https://rustwasm.github.io/docs/wasm-bindgen/\"\u003eThe \u003ccode\u003ewasm-bindgen\u003c/code\u003e Guide\u003c/a\u003e), but they really railroad you into a setup where on top of Cargo you need npm, wasm-pack and webpack. You haven\u0026rsquo;t even run anything yet and you have two package managers and two bundlers!\u003c/p\u003e","title":"Fast Rust to WASM development workflow"},{"content":"You try updating your project to a newer Unreal Engine version, play around for a bit and realize you need to revert to a previous version. You already changed and resaved some of your assets though, what can you do?\nI couldn\u0026rsquo;t find a solution to this after a quick search, but I\u0026rsquo;ve used a hack for this before that could be useful. Note that this is sort of a \u0026ldquo;last resort\u0026rdquo;, and should almost always be a bad idea.\nDISCLAIMER: Please make sure you backup your assets somewhere else before trying this, as this hack could permanently destroy them!\nThe problem For this sample I\u0026rsquo;ll move a static mesh asset that was saved in 4.27 Preview 2 down to 4.26.2, but this should work for any pair of UE4/5 versions. You\u0026rsquo;ll get into more trouble the further apart the engine versions are, though.\nAnyway, if I just try opening the asset saved in 4.27P2 with my 4.26.2 engine and project, I get an error that looks like this:\nLogAssetRegistry: Error: Package E:/Unreal projects/Test_426/Content/BlenderCube.uasset has newer custom version of Dev-Rendering The cause What is a Dev-Rendering? Check DevObjectVersion.cpp. The relevant snippet looks like this:\n// Unique Rendering Object version id const FGuid FRenderingObjectVersion::GUID(0x12F88B9F, 0x88754AFC, 0xA67CD90C, 0x383ABD29); // Register Rendering custom version with Core FDevVersionRegistration GRegisterRenderingObjectVersion(FRenderingObjectVersion::GUID, FRenderingObjectVersion::LatestVersion, TEXT(\u0026#34;Dev-Rendering\u0026#34;)); Here is the start and end of that FRenderingObjectVersion enum:\n// Custom serialization version for changes made in Dev-Rendering stream struct CORE_API FRenderingObjectVersion { enum Type { // Before any version changes were made BeforeCustomVersionWasAdded = 0, // Added support for 3 band SH in the ILC IndirectLightingCache3BandSupport, // Allows specifying resolution for reflection capture probes CustomReflectionCaptureResolutionSupport, RemovedTextureStreamingLevelData, ... // Lots more // Remap Volume Extinction material input to RGB VolumeExtinctionBecomesRGB, // -----\u0026lt;new versions can be added above this line\u0026gt;------------------------------------------------- VersionPlusOne, LatestVersion = VersionPlusOne - 1 }; // The GUID for this custom version number const static FGuid GUID; private: FRenderingObjectVersion() {} }; This enum works like a version tracker for the Dev-Rendering \u0026ldquo;custom version\u0026rdquo;. The idea is that they can have entries in that enum that correspond to changes that affect serialized assets.\nSaved uassets (static meshes, materials, even levels) can get serialized with that GUID we saw before (0x12F88B9F, 0x88754AFC, 0xA67CD90C, 0x383ABD29, which identifies Dev-Rendering), and the current value of FRenderingObjectVersion::LastVersion, which means the asset requires this particular version (or later) of the Dev-Rendering custom version.\nWhenever the asset is deserialized, what it thinks the value for FRenderingObjectVersion::LastVersion is is retrieved from the file and compared with the engine\u0026rsquo;s own value for FRenderingObjectVersion::LastVersion.\nIf the asset has a lower value for LastVersion, it means it was saved with an older engine version (that had less entries in that enum). This is usually not an issue though, because developers can write code on UObject::Serialize(FArchive\u0026amp; Ar) overloads that can automatically upgrade the older asset to the current engine version upon loading, since they know what the serialized representation looked like before and what it should look like now.\nIf the asset has a higher number for LastVersion, it means that the enum had more entries when the asset was saved, i.e. the asset was saved with a newer engine version. The developers can\u0026rsquo;t do much about that unfortunately: The way the asset is represented on disk can have arbitrarily changed in some way our engine doesn\u0026rsquo;t understand yet, so the loading is aborted.\nYou can see where this is going: It\u0026rsquo;s pretty risky to try this if you\u0026rsquo;re trying to downgrade a material and the version for Dev-Rendering changed, but it shouldn\u0026rsquo;t be impossible to manually downgrade something like a static mesh if only the version of FortniteRelease changed, or something like that. Likely our mesh doesn\u0026rsquo;t have any Fortnite data stored in it, so it\u0026rsquo;s serialized representation on disk probably hasn\u0026rsquo;t changed much even though FortniteRelease has. Luckily these GUIDs and values are stored uncompressed/unhashed on the asset files, so we can do something about it.\nThe workaround These version numbers can be tweaked on the assets directly. Here is what the start of that BlenderCube.uasset asset looks like on a hex editor. It doesn\u0026rsquo;t matter what asset (or asset type) it is, this \u0026ldquo;file header\u0026rdquo; should always have more or less the same structure.\nThis header corresponds to a serialized FPackageFileSummary by the way, and you can see the serialization process here.\nAnyway, the highlighted bit is our Dev-Rendering GUID, except that it was serialized as little endian. TL;DR: It will be written to disk inverting the order of bytes (pair of characters) in each of the values in the GUID, so the first group 0x12F88B9F, which has the bytes 12 F8 8B 9F, becomes 9F 8B F8 12, which is the start of the highlighted text on that image.\nThe value in red right after the GUID is the current value for the Dev-Rendering enum: 0000002D, which is hexadecimal for 45. Our asset was saved with FRenderingObjectVersion::LastVersion = 45.\nIf you just switch between the 4.27 and 4.26 branches on the file of the enum\u0026rsquo;s declaration you can see that it has gained a new entry in 4.27: VolumeExtinctionBecomesRGB. This means that in 4.26, LastVersion was probably just 44, which is 2C. We can just write 2C there and save:\nThis should do it. Realistically though, not just Dev-Rendering will change between engine versions, and you\u0026rsquo;d have to repeat this about 5-10 times for different enums.\nThe faster workaround I made a small Python 3 script that should help with the tediousness of this, though. It will make sure that e.g. the value for Dev-Rendering is at most the one that you set it. It has all the custom versions listed on DevObjectVersion.cpp plus the Release custom version, but you may need to manually add others that fit your cases, as individual plugins may also define their own custom version.\nHopefully the script is self-explanatory: The idea is to watch the UE Output Log for errors like we originally saw:\nLogAssetRegistry: Error: Package E:/Unreal projects/Test_426/Content/BlenderCube.uasset has newer custom version of Dev-Rendering Then add entries for those custom versions (here, Dev-Rendering again) with the -1 value within the script\u0026rsquo;s updated_values dict to probe what the value currently is. The script will output something like Read 'Dev-Rendering' with value '45'. Knowing that, you can change the -1 value to 44 and run the script again to downgrade it to 44 instead. Once you run the script the file will be saved and UE will try loading it again, potentially outputting another error like custom version mismatch for FortniteMain. Repeat until it stops complaining.\nThe complete workaround I\u0026rsquo;ve used this a couple times before and it works fine, and it should at least always get your asset to show up on the Content Browser. This is all you need if you\u0026rsquo;re working on UE source and want to downgrade between different changelists, and your build version is still compatible with the asset.\nIf you are moving an asset between different major/minor versions though (like we are doing here by moving it from 4.27P2 to 4.26.2), then we need one extra step.\nAfter you run the script (or did the version changes manually), the asset will show up on the Content Browser. If you double-click the asset though, you\u0026rsquo;ll get a warning like this on the Output Log:\nLogLinker: Warning: Asset \u0026#39;E:/Unreal projects/Test_426/Content/BlenderCube.uasset\u0026#39; has been saved with a newer engine and can\u0026#39;t be loaded. CurrentEngineVersion: 4.26.2-15973114+++UE4+Release-4.26 (Licensee=0). AssetEngineVersion: 4.27.0-16724560+++UE4+Release-4.27 (Licensee=0) The major/minor versions, engine and compatible changelists and Perforce stream names are also saved on the asset, and the engine is trying it\u0026rsquo;s best to prevent us from loading it (as it should, as this is a pretty bad idea in general!).\nBecause of this we need to provide the major/minor/changelist versions to the script as well, so that it can replace that for us. In our case, from that error message we can tell that our asset has major=4, minor=27, changelist=16724560, and our engine has major=4, minor=26 and changelist=15973114. On the script, set update_engine_version to True after you set these values on the variables defined right after it, and then run it.\nThis is it, for real this time! If you can double-click now make sure you save your asset as soon as you can, so that it can be serialized correctly to disk by the engine, hopefully ironing out any small details we could have missed.\nIf this doesn\u0026rsquo;t work for you and your asset just crashes when opening, you may be in deeper trouble unfortunately. The versions could be far enough apart that the serialized representation of your asset changed on disk in some non-trivial way. It\u0026rsquo;s probably not worth it to try and make sense of it, and you should just try recreating/reimporting your asset instead (or find an even crazier hack :) ).\n","permalink":"https://1danielcoelho.github.io/downgrade-unreal-asset/","summary":"\u003cp\u003eYou try updating your project to a newer Unreal Engine version, play around for a bit and realize you need to revert to a previous version. You already changed and resaved some of your assets though, what can you do?\u003c/p\u003e","title":"How to downgrade Unreal Engine assets"},{"content":"I have a long-term goal of making a GAN that is capable of generating songs similar to the provided training data, mostly as a learning exercise. The idea would be to operate on waveforms directly using convolution, instead of deferring to MIDI approaches.\nIn the end I imagine I\u0026rsquo;ll try replicating something like jukebox from OpenAI using VQ-VAE, but for now I\u0026rsquo;m sticking to DCGAN-like model, except that 1d.\nFor this article we\u0026rsquo;ll start a lot smaller and just try to get a GAN to generate a Gaussian curve. The training data will be 10k different 100-sample .wav files that all contain a single centered Gaussian curve with some normal noise. I\u0026rsquo;m going to be using Pytorch and torchaudio.\nThe models Here is what my generator looks like:\nclass Generator(nn.Module): def __init__(self, nz, ngf): super(Generator, self).__init__() assert(ngf % 32 == 0 and ngf \u0026gt;= 32) self.main = nn.Sequential( nn.ConvTranspose1d( in_channels=nz, out_channels=ngf, kernel_size=4, stride=1, padding=0, dilation=1, bias=False, ), nn.BatchNorm1d(ngf), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.ConvTranspose1d( in_channels=ngf, out_channels=ngf // 2, kernel_size=4, stride=2, padding=0, dilation=1, bias=False, ), nn.BatchNorm1d(ngf // 2), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.ConvTranspose1d( in_channels=ngf // 2, out_channels=ngf // 4, kernel_size=4, stride=2, padding=0, dilation=1, bias=False, ), nn.BatchNorm1d(ngf // 4), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.ConvTranspose1d( in_channels=ngf // 4, out_channels=ngf // 8, kernel_size=4, stride=2, padding=0, dilation=1, bias=False, ), nn.BatchNorm1d(ngf // 8), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.ConvTranspose1d( in_channels=ngf // 8, out_channels=1, kernel_size=10, stride=2, padding=0, dilation=1, bias=False, ), nn.Tanh(), ) def forward(self, input): return self.main(input) And here is what my discriminator looks like:\nclass Discriminator(nn.Module): def __init__(self, ndf): super(Discriminator, self).__init__() assert(ndf % 16 == 0 and ndf \u0026gt;= 16) self.main = nn.Sequential( nn.Conv1d( in_channels=1, out_channels=(ndf // 16), kernel_size=4, stride=1, padding=0, dilation=1, bias=False ), nn.BatchNorm1d(ndf // 16), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.Conv1d( in_channels=(ndf // 16), out_channels=(ndf // 8), kernel_size=4, stride=2, padding=0, dilation=1, bias=False ), nn.BatchNorm1d(ndf // 8), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.Conv1d( in_channels=(ndf // 8), out_channels=(ndf // 4), kernel_size=4, stride=2, padding=0, dilation=1, bias=False, ), nn.BatchNorm1d(ndf // 4), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.Conv1d( in_channels=(ndf // 4), out_channels=(ndf // 2), kernel_size=4, stride=2, padding=0, dilation=1, bias=False ), nn.BatchNorm1d(ndf // 2), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.Conv1d( in_channels=(ndf // 2), out_channels=ndf, kernel_size=4, stride=2, padding=0, dilation=1, bias=False ), nn.BatchNorm1d(ndf), nn.LeakyReLU(0.2, inplace=True), nn.Dropout(p=0.2), nn.Conv1d( in_channels=ndf, out_channels=1, kernel_size=4, stride=2, padding=0, dilation=1, bias=False ), ) def forward(self, input): return self.main(input) The goal was to maintain the general idea of strided convolutions of the the DCGAN architecture while incorporating some recommendations from other posts and resources like ganhacks. I guess the main differences are:\nLeakyReLU layers to prevent vanishing gradients; Dropout to prevent overfitting; Removal of the last Sigmoid layer of the discriminator (paired with the usage of nn.BCEWithLogitsLoss() instead of nn.BCELoss()) to better handle mode collapse, as this effectively prevents the discriminator from getting stuck at zero loss. You may ocasionally see some weird values for kernel_size, stride, padding and dilation, but the performance of the model shouldn\u0026rsquo;t be too affected by tiny details like these, and these are the easiest buttons to push when trying to match the input/output dimension of the data.\nData loader Here is what my Dataset look like. Nothing fancy: Just glob all .wav files form a folder.\nfrom pathlib import Path import torch import torchaudio from torch.utils.data import Dataset def collect_files(folder_root, formats): result = [] for fmt in formats: for path in Path(folder_root).rglob(\u0026#39;*.\u0026#39; + fmt): result.append(str(path)) return result class WavDataset(Dataset): def __init__(self, root_dir, formats=[\u0026#39;wav\u0026#39;]): self.files = collect_files(root_dir, formats) def __len__(self): return len(self.files) def __getitem__(self, idx): waveform, _ = torchaudio.load(self.files[idx]) return waveform Main training script The actual program looks like this:\ndevice = \u0026#34;cpu\u0026#34; if torch.cuda.is_available(): device = \u0026#34;cuda\u0026#34; torch.cuda.empty_cache() # Hyperparameters instance_to_resume = 13 num_epochs = 5000 batch_size = 100 learning_rate = 5e-5 beta1 = 0.9 nz = 100 # Number of values (\u0026#34;features\u0026#34;) in the noise supplised to generator ngf = 512 # Number of generator feature maps (how many channels it will generate for each noise feature) ndf = 512 # Number of discriminator feature maps real_label = 1 fake_label = 0 hard_real_labels = torch.full((batch_size,), real_label, dtype=torch.float32, device=device, requires_grad=False) hard_fake_labels = torch.full((batch_size,), fake_label, dtype=torch.float32, device=device, requires_grad=False) # Load all of our songs dataset = WavDataset(os.path.join(\u0026#39;D:\u0026#39;, \u0026#39;AI\u0026#39;, \u0026#39;Wavegen\u0026#39;, \u0026#39;data\u0026#39;, \u0026#39;phantom_100\u0026#39;)) num_samples = len(dataset) print(f\u0026#34;Detected {num_samples} tracks to use\u0026#34;) # Create loader data_loader = torch.utils.data.DataLoader( dataset=dataset, batch_size=batch_size, shuffle=True ) # Create models generator = Generator(nz, ngf).to(device) discriminator = Discriminator(ndf).to(device) # Create optimizers generator_optimizer = torch.optim.Adam(generator.parameters(), lr=learning_rate, betas=(beta1, 0.999)) discriminator_optimizer = torch.optim.Adam(discriminator.parameters(), lr=learning_rate, betas=(beta1, 0.999)) # Create loss function loss_func = nn.BCEWithLogitsLoss() # Train for epoch in range(num_epochs): for i, data in enumerate(data_loader): actual_batch_size = data.size()[0] sample_length = data.size()[2] real_batch = data.to(device) # Soften labels (probably don\u0026#39;t need to do this every batch as the loader shuffles anyway) real_labels = torch.randn_like(hard_real_labels) * 0.10 + hard_real_labels fake_labels = torch.randn_like(hard_fake_labels) * 0.10 + hard_fake_labels # Train generator generator_optimizer.zero_grad() noise = torch.randn(actual_batch_size, nz, 1, device=device) fake_batch = generator(noise) fake_output = discriminator(fake_batch).view(-1) generator_loss = loss_func(fake_output, real_labels[:actual_batch_size]) generator_loss.backward() generator_optimizer.step() # Train discriminator discriminator_optimizer.zero_grad() real_output = discriminator(real_batch).view(-1) fake_output = discriminator(fake_batch.detach()).view(-1) real_loss = loss_func(real_output, real_labels[:actual_batch_size]) fake_loss = loss_func(fake_output, fake_labels[:actual_batch_size]) discriminator_loss = fake_loss + real_loss discriminator_loss.backward() discriminator_optimizer.step() # Keep statistics gen_grad_param_norms = [] for param in generator.parameters(): gen_grad_param_norms.append(param.grad.norm()) mean_gen_grad_norm = torch.tensor(gen_grad_param_norms).mean().item() disc_grad_param_norms = [] for param in discriminator.parameters(): disc_grad_param_norms.append(param.grad.norm()) mean_disc_grad_norm = torch.tensor(disc_grad_param_norms).mean().item() mean_real_output = real_output.mean().item() mean_fake_output = fake_output.mean().item() if i % 10 == 0: print(f\u0026#34;Epoch [{epoch+1}/{num_epochs}], Batch [{i+1}/{num_samples // batch_size}] loss_g: {generator_loss:.4f}, loss_d: {discriminator_loss:.4f}, mean_real_output: {mean_real_output:.4f}, mean_fake_output: {mean_fake_output:.4f}, mean_gen_grad_norm: {mean_gen_grad_norm:.4f}, mean_disc_grad_norm: {mean_disc_grad_norm:.4f}\u0026#34;) I opted for the Adam optimizer with very slow training rates as these data are only 100-samples long, so it should be quick to train anyway.\nWe\u0026rsquo;re using soft labels in order to try and make things a bit more difficult for the Discriminator. The point is that if the discriminator starts getting everything right all the time there will be no gradients to train with, and no progress. On top of that, along with the usage of Dropout layers, soft labels force the discriminator to be more robust.\nNote that we run the discriminator on fake_batch twice. As far as I can tell this is actually the right thing to do: When training the generator from generator_loss.backward() we need the errors to flow back from fake_output all the way to generator, so we obviously need to call discriminator(fake_batch). When training the discriminator from discriminator_loss.backward() we also need the errors to flow from fake_loss through the discriminator, but the loss function being used is different, so the errors and gradients at the discriminator will be different. Pytorch really doesn\u0026rsquo;t seem to allow direct manipulation of the computation graphs (1, 2), so it means we need to run the discriminator again. We can at least detach fake_batch there though, as we don\u0026rsquo;t care about those errors flowing back up to the generator.\nThere are entirely different ways to organize a GAN training loop that avoids this double call to discriminator(fake_batch) (check \u0026ldquo;strategy 2\u0026rdquo; on this very helpful reference), but the trade-off involves retaining the computational graph (with discriminator_loss.backward(retain_graph=True)) between generator and discriminator training steps, which can lead to higher memory usage. In my case the batch size is the limiting factor in the training process as a whole, so using that approach actually led to a loss of performance.\nResults Let\u0026rsquo;s look at some results from some successful trainings.\nYou may have noticed that the training loop prints out some data. Here is how that looks plotted on graphs:\nHere\u0026rsquo;s what these plots mean:\nThe topmost plot just shows the generator and discriminator losses; The one in the middle shows the average output for the discriminator for real and fake outputs. I\u0026rsquo;ve opted for classifying based on logits instead of probability values (so as to prevent the discriminator from getting stuck at zero loss), which means that the discriminator should output 1 if it considers a track real, and 0 if it considers it fake, but it is free to output arbitrary values like 100.0 if it really thinks a track is real or -23947.0 if it really really thinks a track is fake. The value of that second plot then describes the mean of these outputs for a given batch; The plot on the bottom describes the average magnitude of the gradients for all weights in the generator and discriminator, and is a rough measurement of \u0026ldquo;how much the weights changed\u0026rdquo; for a given batch, i.e. \u0026ldquo;how much it learned. You may notice that fake_output is basically a rescaled and flipped version of G loss here by the way, but this is just an artifact of our choice of loss functions.\nThe most interesting aspect of this for me was that I always figured that during the training process I\u0026rsquo;d see a gradual improvement of my output as time went on. Not once did this happen. There is always a very harsh transition where the model goes from outputting garbage to outputting something most of the way there.\nHere is what sample generator outputs (red) look like when compared to the ideal output (blue) during training, at batch 200 and onwards:\nYou can see that something really interesting happens right around batch 900, which is right about where we see those harsh transitions on the progress plot. I\u0026rsquo;m still a beginner at deep learning and GANs, so I\u0026rsquo;m not entirely sure what exactly, but please let me know if you have any ideas.\nAnother interesting thing I\u0026rsquo;ve noticed is that GANs seem to reach a minimum loss at some point, and then progressively diverge, in some ways. Here is an entirely different training process (with all the same parameters) that was left training for way too long:\nAgain, \u0026ldquo;the transition\u0026rdquo; happens around batch 900, but you can see that the generator loss tends to slowly increase after having reached a minimum near batch 4000 or so. The discriminator loss tends to decrease over time too, with the average outputs for real and fake samples diverging in score, which is a bad sign: Ideally the discriminator outputs should converge to the same value as the generated samples become more and more similar to the training data, which doesn\u0026rsquo;t happen here.\nAlso, as far as I can tell the fact that the norm of generator/discriminator weights also never seem to taper off suggests that it\u0026rsquo;s either learning too fast or that it\u0026rsquo;s beginning to overfit the training data.\nConclusion It was pretty easy to find some references/recommendations on how to write a Pytorch GAN, but I struggled to find references that helped me make sense of the reasoning behind some of those choices, and also had trouble finding references that helped me debug the training process as a whole. Hopefully this post helps in that area, and shows a bit how a GAN training process should feel like.\nIn future posts I really want to investigate what is happening to these weights during \u0026ldquo;the transition\u0026rdquo;, and also measure the impact of some of those choices, like Dropout layers and hyperparameter values, so stay tuned!\nReferences https://github.com/soumith/ganhacks https://openai.com/blog/jukebox/ https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html https://www.fatalerrors.org/a/detach-when-pytorch-trains-gan.html ","permalink":"https://1danielcoelho.github.io/1d-dcgan-transition/","summary":"\u003cp\u003eI have a long-term goal of making a GAN that is capable of generating songs similar to the provided training data, mostly as a learning exercise. The idea would be to operate on waveforms directly using convolution, instead of deferring to MIDI approaches.\u003c/p\u003e","title":"1D DCGAN for waveform generation"},{"content":"So this issue cost me about a day of my life. Hopefully you get to this post before the problem wastes your time too.\nThe FTransform struct in UE is a decomposed transform representation, which means it internally just contains an FQuat rotation, an FVector translation and a FVector scale. This is in contrast with FMatrix, which is just a regular 4x4 matrix that can be used as a transformation.\nOne major, sneaky problem with this is that you may run into trouble when using non-uniform scaling (i.e. the scale vector has different values for two or more components, like [3.0, 1.0, 1.0]).\nFirst, let\u0026rsquo;s have a look what happens when all you have is uniform scaling:\nFTransform A{ FRotator{}, // Rotation FVector{1.0f, 2.0f, 3.0f}, // Translation FVector{2.0f, 2.0f, 2.0f} // Scale }; FTransform AInv = A.Inverse(); FTransform AMaybeIdentity = A * AInv; FVector Pos = FVector{ 50.0f, 60.0f, 70.0 }; FVector PosAfterA = A.TransformPosition( PosA ); FVector PosAfterAInv = AInv.TransformPosition( PosAfterA ); Print all this out and you get something like this:\nA: Trans: [1.0, 2.0, 3.0]; Rot: [0.0, 0.0, 0.0, 1.0]; // Quaternions btw Scale: [2.0, 2.0, 2.0] AInv: Trans: [-1.0, -2.0, -3.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [0.5, 0.5, 0.5] AMaybeIdentity: Trans: [0.0, 0.0, 0.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [1.0, 1.0, 1.0] Pos: [50.0, 60.0, 70.0] PosAfterA: [101.0, 122.0, 143.0] PosAfterAInv: [50.0, 60.0, 70.0] Nothing weird here. AMaybeIdentity is actually the identity, and transforming Pos with A and then AInv gets us back to Pos. Let\u0026rsquo;s use non-uniform scaling instead:\nFTransform B{ FRotator{}, // Rotation FVector{ 1.0f, 2.0f, 3.0f }, // Translation FVector{ 2.0f, 1.0f, 1.0f } // Scale }; FTransform BInv = B.Inverse(); FTransform BMaybeIdentity = B * BInv; FVector Pos = FVector{ 50.0f, 60.0f, 70.0 }; FVector PosAfterB = B.TransformPosition( PosB ); FVector PosAfterBInv = BInv.TransformPosition( PosAfterB ); Print the B case we get this:\nB: Trans: [1.0, 2.0, 3.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [2.0, 1.0, 1.0] BInv: Trans: [-0.5, -2.0, -3.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [0.5, 1.0, 1.0] BMaybeIdentity: Trans: [0.0, 0.0, 0.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [1.0, 1.0, 1.0] Pos: [50.0, 60.0, 70.0] PosAfterB: [101.0, 62.0, 73.0] PosAfterBInv: [50.0, 60.0, 70.0] \u0026hellip;Oh? It still seems to work fine. I guess I don\u0026rsquo;t need to pay attention to non-uniform scalings after all, right?\nLet\u0026rsquo;s just add a small rotation:\nFTransform C{ FRotator{ 10.0f, 20.0f, 30.0f }, // Rotation FVector{ 1.0f, 2.0f, 3.0f }, // Translation FVector{ 2.0f, 1.0f, 1.0f } // Scale }; FTransform CInv = C.Inverse(); FTransform CMaybeIdentity = C * CInv; FVector Pos = FVector{ 50.0f, 60.0f, 70.0 }; FVector PosAfterC = C.TransformPosition( PosC ); FVector PosAfterCInv = CInv.TransformPosition( PosAfterC ); This is what we get now:\nC: Trans: [1.0, 2.0, 3.0]; Rot: [-0.239298329, -0.127679437, 0.144878119, 0.951548517]; Scale: [2.0, 1.0, 1.0] CInv: Trans: [-1.65730095 -0.102469981 -3.23926759]; Rot: [0.239298329, 0.127679437, -0.144878119, 0.951548517]; Scale: [0.5, 1.0, 1.0] CMaybeIdentity: Trans: [0.0, 0.0, 0.0]; Rot: [0.0, 0.0, 0.0, 1.0]; Scale: [1.0, 1.0, 1.0] Pos: [50.0, 60.0, 70.0] PosAfterC: [58.8023224, 115.580849, 50.5213852] PosAfterCInv: [73.2543716, 66.2024841, 79.0265503] \u0026lt;--- !!! Not great.\nCheck how sneaky this is! CMaybeIdentity is still actually an identity. Only when you transform a point with C and then CInv that you see the problem: PosAfterCInv != Pos.\nThe actual problem happens when you apply the inverse. To understand what\u0026rsquo;s going on, have a look at what FTransform::TransformPosition and FTransform::Inverse look like (roughly):\nFVector FTransform::TransformPosition(const FVector\u0026amp; V) const { return Rotation.RotateVector( Scale3D * V ) + Translation; } FTransform FTransform::Inverse() const { FQuat InvRotation = Rotation.Inverse(); FVector InvScale3D = GetSafeScaleReciprocal( Scale3D ); FVector InvTranslation = InvRotation * ( InvScale3D * -Translation ); return FTransform( InvRotation, InvTranslation, InvScale3D ); } The root of the problem is that FTransform::TransformPosition always scales, then rotates, then translates, whether you\u0026rsquo;re applying a transform or the inverse of a transform.\nIgnore the translation for now: If you only have uniform scaling, then inverting the transform by applying the inverse scaling and then the inverse rotation is not a problem: It doesn\u0026rsquo;t matter what rotation you applied to the object, since you\u0026rsquo;ll scale it uniformly anyway.\nThe problem is that if you have non-uniform scaling, applying the inverse transform will first apply the inverse scale and only later inverse the rotation. This means the inverse scaling will happen around the rotated axes!\nTo invert it properly using a decomposed representation you would need to do the steps in the reverse order: First apply the reverse translation, then apply the reverse rotation, and finally apply the reverse scale, which is not something that FTransform::TransformPosition will do.\nIn UE you have 3 practical ways around this problem:\nApply the inverse translation, rotation and scaling manually in that order;\nUse FTransform::InverseTransformPosition, which does exactly what we want:\nFVector FTransform::InverseTransformPosition(const FVector \u0026amp;V) const { return (Rotation.UnrotateVector(V - Translation)) * GetSafeScaleReciprocal(Scale3D); } Or get the FMatrix from your transform, invert that, and transform your points with the inverted matrix instead: FMatrix CMatInv = C.ToMatrixWithScale().Inverse(); FVector CorrectPos = CMatInv.TransformPosition( PosAfterC ); // [50.0, 60.0, 70.0]; In conclusion, try to keep this annoyance in mind: This is the sort of thing that you learn about, forget it, and then get hit by it for critical damage later, like what happened to me. Maybe I\u0026rsquo;ll remember this now that I\u0026rsquo;ve blogged about it.\nTL;DR: Avoid using FTransform if you have non-uniform scalings and need the inverse transform, or pay very close attention to what you\u0026rsquo;re doing!\nThanks for reading!\n","permalink":"https://1danielcoelho.github.io/unreal-non-uniform-scaling-gotcha/","summary":"\u003cp\u003eSo this issue cost me about a day of my life. Hopefully you get to this post before the problem wastes your time too.\u003c/p\u003e\n\u003cp\u003eThe \u003ccode\u003eFTransform\u003c/code\u003e struct in UE is a decomposed transform representation, which means it internally just contains an \u003ccode\u003eFQuat\u003c/code\u003e rotation, an \u003ccode\u003eFVector\u003c/code\u003e translation and a \u003ccode\u003eFVector\u003c/code\u003e scale. This is in contrast with \u003ccode\u003eFMatrix\u003c/code\u003e, which is just a regular 4x4 matrix that can be used as a transformation.\u003c/p\u003e","title":"Gotcha when using FTransforms in Unreal Engine"},{"content":"This is going to be the first of a few of blog posts detailing some stuff I\u0026rsquo;ve learned doing deep dives in the Unreal Engine source.\nThis is mostly to get the ball rolling as far as this blog is concerned (as working with Unreal is a large part of my day job and should be the easiest to write about), but also because UE\u0026rsquo;s documentation is not the best, and there\u0026rsquo;s a surprising shortage of posts like this out there. It seems like the community just assumes everyone is forced to dig through the code, and while there\u0026rsquo;s nothing fundamentally wrong with reading code, it does get in the way a bit when all you want is an overview. It also sucks that everyone has to rediscover the same insights over and over, so hopefully this can provide a net saving of man hours to the world.\nThis post is about UE 4.26 in particular, but most of these details probably haven\u0026rsquo;t changed much since UE 3.\nI will sometimes link straight to the UE source on Github at some points. If you don\u0026rsquo;t have access to it yet, you can get it for free in just a couple of seconds by following this guide.\nUObjects Let\u0026rsquo;s get into it then! The purpose of this post is to explain how UObject, UClass, UBlueprint, UBlueprintGeneratedClass and other concepts like the Class Default Object all interact. Hopefully this post works as sort of a crash course on the base classes of the engine.\nLet\u0026rsquo;s start off with a simple, pure C++ class. I\u0026rsquo;m using a TArray data member here, but that\u0026rsquo;s just an analogue for std::vector, so there\u0026rsquo;s nothing special there.\nclass MyObject { public: float Multiply(float OtherValue) { return MyValue * OtherValue; } float MyValue = 2.3f; TArray\u0026lt;double\u0026gt; MyValueArray; float UnAnnotatedValue = 3.0f; }; In order to see MyValue and MyValueArray in the editor, interact with our class via blueprints or even create a blueprint class that derives MyObject, we need some changes:\n#include \u0026#34;UObject/ObjectMacros.h\u0026#34; #include \u0026#34;MyObject.generated.h\u0026#34; UCLASS( BlueprintType ) class UMyObject : public UObject { GENERATED_BODY() public: float Multiply(float OtherValue) { return MyValue * OtherValue; } float MyValue = 2.3f; TArray\u0026lt;int32\u0026gt; MyValueArray; float UnAnnotatedValue = 3.0f; }; A few things happened here: We derived from UObject, which is the base class for objects managed by Unreal and is required to get our class garbage collected, replicated over the network, serialized, and more. We also had to rename our class to UMyObject, as that is the naming convention for classes that derive from UObject. We also added a couple of includes, and a couple of macros, which are required to get Unreal Header Tool (UHT) to automatically generate some code for us when we compile.\nNote that these macros just expand to more macros, and won\u0026rsquo;t help us understand what\u0026rsquo;s going on (although you can peek at the UCLASS macro if you want). Instead, these macros work more as annotations: UHT will parse this code, see those anotations (like UCLASS()), and know that it needs to generate some code about that class and place it somewhere (and obviously remove the annotations afterwards).\nPart of that generated code goes in that \u0026quot;MyObject.generated.h\u0026quot; file, and part of that code is injected in the location of that GENERATED_BODY() macro just before we compile. You will likely never need to interact with the \u0026quot;*.generated.h\u0026quot; files though, and that\u0026rsquo;s good, because there\u0026rsquo;s some pretty crazy auto-generated code in there.\nYou can provide some optional arguments to some of these annotations, like how we did UCLASS( BlueprintType ). That one in particular allows our UMyObject objects to interact with blueprints, but you can check the source for all the optional arguments you can provide for UCLASS.\nIf you\u0026rsquo;re interested about this part, you can find some more details about reflection on this excellent post by Michael Noland, and on the official doc page for the unreal UObjects.\nUClasses An important aspect of annotating our class for UHT like this is that it leads to a UClass object being generated for our UMyObject during engine initialization (both in the editor and for a packaged game). We also get a static function UMyObject::StaticClass() automatically generated and injected into our class definition, so we can do this to get a UClass object:\nUClass* MyObjectsClass = UMyObject::StaticClass(); Warning: Don\u0026rsquo;t get confused here! These UClass objects themselves derive from UObject, and so are garbage collected, reflected, and managed just like instances of our UMyObject. This is how you can still manipulate classes directly in the engine, like providing classes to blueprint nodes and creating properties of \u0026ldquo;Class\u0026rdquo; type (we will do this later!).\nGetting back on track, that UClass instance we got there holds tons of useful data about our UMyObject type, like whatever data members and functions our class has that the engine can reason about.\nBy the way, yes, there is a UClass for UClass objects as well! Rest assured you will rarely ever need to consider this, so you may ignore all of that for now and just consider UClass as a base class.\nIf we checked that UClass* MyObjectsClass object now we wouldn\u0026rsquo;t find information about our data members and functions though. We need some more of those annotation macros for that:\n#include \u0026#34;UObject/ObjectMacros.h\u0026#34; #include \u0026#34;MyObject.generated.h\u0026#34; UCLASS() class UMyObject : public UObject { GENERATED_BODY() public: UFUNCTION( BlueprintCallable ) float Multiply(float OtherValue) { return MyValue * OtherValue; } UPROPERTY( EditAnywhere ) float MyValue = 2.3f; UPROPERTY( EditAnywhere ) TArray\u0026lt;int32\u0026gt; MyValueArray; float UnAnnotatedValue = 3.0f; }; I\u0026rsquo;ve added the UFUNCTION and UPROPERTY() annotations with some useful self-explanatory optional arguments this time. You can check the source for a list with all the optional arguments for UFUNCTION, and for the optional arguments for UPROPERTY.\nNow that our members are annotated, we could iterate our UClass\u0026rsquo;s fields and actually get info on our data members, like this:\nUClass* MyObjectsClass = UMyObject::StaticClass(); for ( TFieldIterator\u0026lt;FProperty\u0026gt; PropertyIterator( MyObjectsClass ); PropertyIterator; ++PropertyIterator ) { FProperty* Property = *PropertyIterator; // Would log \u0026#34;MyValue\u0026#34; and \u0026#34;MyValueArray\u0026#34; UE_LOG( LogTemp, Log, TEXT( \u0026#34;%s\u0026#34; ), *Property-\u0026gt;GetName() ); } for ( TFieldIterator\u0026lt;UFunction\u0026gt; FunctionIterator( MyObjectsClass ); FunctionIterator; ++FunctionIterator ) { UFunction* Func = *FunctionIterator; // Would log Multiply UE_LOG( LogTemp, Log, TEXT( \u0026#34;%s\u0026#34; ), *FunctionIterator-\u0026gt;GetName() ); } We can of course get tons of more useful information about those properties and functions, including their value size, metadata. This is the actual \u0026ldquo;reflection\u0026rdquo; capability I mentioned previously: Objects of UMyObject can query what data members and functions they own, and read/write to them.\nClass default objects One very important member of the UClass instance we got there is the Class Default Object (CDO). This object is just another instance of our UMyObject class, but it is owned by the UClass directly. It will hold the default values for our properties, and in some contexts it is used as a template for all other created instances. These CDOs are used everywhere throughout the engine, and have some surprising uses.\nFor example, configuration and \u0026ldquo;options\u0026rdquo; container objects in Unreal are usually just UObjects too, and the properties are the actual options that you can set. When you edit them on the editor, what you\u0026rsquo;re doing is editing the CDO object\u0026rsquo;s values for those properties, and those values can be saved to disk and read back again later.\nTake for example the UBlueprintEditorSettings class. It contains the options you see under \u0026ldquo;Blueprint Editor\u0026rdquo; on the Edit -\u0026gt; Editor Preferences window.\nThat class is defined at Engine\\Source\\BlueprintGraph\\Public\\BlueprintEditorSettings.h and as of 4.26 looks like this:\nUCLASS(config=EditorPerProjectUserSettings) class BLUEPRINTGRAPH_API UBlueprintEditorSettings :\tpublic UObject { GENERATED_UCLASS_BODY() // Style Settings public: /** Should arrows indicating data/execution flow be drawn halfway along wires? */ UPROPERTY(EditAnywhere, config, Category=VisualStyle, meta=(DisplayName=\u0026#34;Draw midpoint arrows in Blueprints\u0026#34;)) bool bDrawMidpointArrowsInBlueprints; /** Determines if lightweight tutorial text shows up at the top of empty blueprint graphs */ UPROPERTY(EditAnywhere, config, Category = VisualStyle) bool bShowGraphInstructionText; /** If true, fade nodes which are not connected to the selected nodes */ UPROPERTY(EditAnywhere, config, Category = VisualStyle) bool bHideUnrelatedNodes; /** If true, use short tooltips whenever possible */ UPROPERTY(EditAnywhere, config, Category = VisualStyle) bool bShowShortTooltips; // A lot more stuff below There is some code somewhere that automatically prettifies the variable names before showing it in the editor, so something like bHideUnrelatedNodes becomes Hide Unrelated Nodes automatically. You can override that and get it to show something else when viewed in the Editor by using the UPROPERTY argument meta=(DisplayName=\u0026quot;Something else\u0026quot;) though.\nThe cool thing is that at any point in your C++ code, if you wanted to get or set the value of bHideUnrelatedNodes for whatever reason, you can just do this:\nUBlueprintEditorSettings* Settings = GetMutableDefault\u0026lt;UBlueprintEditorSettings\u0026gt;(); Settings-\u0026gt;bHideUnrelatedNodes = false; Settings-\u0026gt;SaveConfig(); GetMutableDefault\u0026lt;T\u0026gt;() is just a convenience around T::StaticClass()-\u0026gt;GetDefaultObject() by the way.\nAnother interesting bit is that UCLASS(config=EditorPerProjectUserSettings) annotation on top of UBlueprintEditorSettings. That means that the engine will serialize the CDO of that class to \u0026lt;ProjectName\u0026gt;\\Saved\\Config\\\u0026lt;Platform\u0026gt;\\EditorPerProjectUserSettings.ini whenever you call that SaveConfig() (which is a function defined directly on UObject by the way). This is what the corresponding part of that ini file looks like:\n... [/Script/BlueprintGraph.BlueprintEditorSettings] bDrawMidpointArrowsInBlueprints=False bShowGraphInstructionText=True bHideUnrelatedNodes=False bShowShortTooltips=True ... Debugging CDOs Let\u0026rsquo;s have a look at those CDOs and UClass objects in practice. In order to make it easy for us to analyze our objects, I\u0026rsquo;ll make a quick actor that I can place on the level and interact with. It doesn\u0026rsquo;t matter much for this post, but if you want to follow along it looks like this:\n#include \u0026#34;UObject/ObjectMacros.h\u0026#34; #include \u0026#34;GameFramework/Actor.h\u0026#34; #include \u0026#34;MyActor.generated.h\u0026#34; UCLASS( Blueprintable ) class AMyActor : public AActor { GENERATED_BODY() public: UFUNCTION( BlueprintCallable ) void ReceiveMyObject( UMyObject* Object ); }; Here is what the implementation of that one function looks like:\n#include \u0026#34;MyActor.h\u0026#34; void AMyActor::ReceiveMyObject( UMyObject* Object ) { UClass* StaticClass = UMyObject::StaticClass(); UClass* Class = Object-\u0026gt;GetClass(); UMyObject* CDO = Class-\u0026gt;GetDefaultObject\u0026lt;UMyObject\u0026gt;(); // Get all the instances that use this CDO TArray\u0026lt;UObject*\u0026gt; InstancesOfCDO; CDO-\u0026gt;GetArchetypeInstances(InstancesOfCDO); } I\u0026rsquo;ve placed a breakpoint at the end of ReceiveMyObject, so we can have a look at what happens when we give this function an instance of UMyObject:\nFirst of all, as a sanity check we can confirm (underlined in red) that Object-\u0026gt;GetClass() == UMyObject::StaticClass(). That is also the same thing we get if we drill down to the ClassPrivate field of Object. We can also see that we can use GetArchetypeInstances to find all objects that have CDO as a default in any way, and it found the same object that we received (underlined in green). \u0026ldquo;Archetype\u0026rdquo; here just roughly means that the object can be used as a template. There\u0026rsquo;s more nuance to that, but we\u0026rsquo;ll explore this in more detail in a future post.\nAlso note that the CDO for the UMyObject class has the values that it got from our C++ default member initializers (e.g. when we wrote float MyValue = 2.3f; directly on the class declaration). You could use the regular C++ constructor to initialize those values if you want, though. The point is that the CDO is just another regular instance of that class.\nBy the way, on the visual scripting side, you can use the GetClassDefaults blueprint node to read (but not set) the property values on the CDO.\nIf you doubted me before about the UClass holding the reflected data about members, have a look at this:\nI\u0026rsquo;ve expanded UMyObject\u0026rsquo;s UClass. We can clearly in red where it holds information about our Multiply UFunction, as well as our MyValue float property and our MyValueArray array property. It stores it in a linked list, and there\u0026rsquo;s no sign of our UnAnnotatedValue, since it wasn\u0026rsquo;t annotated. It would have been pointed to by the Next pointer in green if it were annotated though.\nOne last important thing you should know about CDOs: It\u0026rsquo;s very likely that they will be constructed during engine initialization along with their owner UClass objects, where a lot of other things aren\u0026rsquo;t fully initialized. The CDO is otherwise just a regular instance of our UMyObject though, and it will call the regular C++ constructor when being created, if you have one defined. This means that whatever we put in our constructors for UObject-derived classes like our UMyObject shouldn\u0026rsquo;t really expect much from its context or try using other classes too much, as they may not have been initialized yet.\nIf it\u0026rsquo;s unavoidable, you can usually check if that particular instance of UMyObject is a CDO or not by calling UObject::IsTemplate(), and not doing your context-dependent initialization in that case. Something like this:\nUMyObject::UMyObject() { if( !IsTemplate() ) { // Do things that may not work during engine initialization } } UBlueprints Objects of our UMyObject class can be manipulated by the engine now: Instances of those would be replicated, serialized and garbage collected like you\u0026rsquo;d expect, and can interact with the many different subsystems in Unreal, like Niagara and the Sequencer and whatever. This could be all you need, especially if whatever you\u0026rsquo;re building is more on the C++ side.\nIf you\u0026rsquo;re building something more on the visual scripting side, then you\u0026rsquo;ll likely want to create blueprint functions on your UMyObject, and have it interact with your level and other blueprints that way. We can accomplish that by creating a blueprint class that derives from the UMyObject class, by first clicking Add/Import, picking Blueprint Class\u0026hellip;\n\u0026hellip; and then choosing our UMyObject object as a base class (note that the U prefix is dropped here, as well as through most of the Editor).\nI\u0026rsquo;ll name our derived blueprint class \u0026ldquo;DerivedObject\u0026rdquo;, so we now get a new asset on our content browser that looks like this:\nThis asset is a UBlueprint asset, also known as a Blueprint Class. This is not a UClass, nor a CDO, nor an instance of DerivedObject, it is something entirely different. If you double-click this it will open a Blueprint Editor, that looks like this:\nThe red arrow points to the base class: In our case it corresponds to our UMyObject class. If you click on the Class Defaults button pointed to in orange, the area on the right (pointed to in green) will display some property values.\nWhat you\u0026rsquo;re looking at on the right are the values of DerivedObject\u0026rsquo;s CDO\u0026rsquo;s properties, which on the blueprint/visual scripting side are usually referred to as \u0026ldquo;Class Defaults\u0026rdquo;. In particular, you can see a UMyObject section for the properties that the DerivedObject class gets by deriving UMyObject, i.e. My Value and My Value Array (it prettified our property names here too, like it did for UBlueprintEditorSettings). If we create a new variable on our DerivedObject class (which we will do later), the CDO\u0026rsquo;s value for it would show up here, in another section.\nNote how we never set these values before: They are initially set with whatever the defaults are on the base class (i.e. UMyObject for us). Defaults are specific to each class though, so we could set this to 5.0f, and that value would be used for DerivedObject instances, while 2.3f would still be used for UMyObject instances.\nAnother useful thing to know is that if you have a few instances of DerivedObject out there with MyValue == 2.3f, and using this editor you change the default from 2.3f to 2.5f, all of those instances would be updated too. Their properties would not be updated if their values differed from the CDO\u0026rsquo;s by the time the CDO\u0026rsquo;s changed, though, so you get to keep your manually set values if you have them.\nLet\u0026rsquo;s have a look at what happens when we provide a DerivedObject to our ReceiveMyObject function from before. Remember, UMyObject is a parent class of DerivedObject, and because our parameter is just a pointer to the base class, we can receive a DerivedObject with no changes to our function.\nCheck it out, Object\u0026rsquo;s class, and it\u0026rsquo;s CDO class are no longer the same as UMyObject::StaticClass(), they\u0026rsquo;re something else now. If you look at the type (far right on the lines underlined in green) you\u0026rsquo;ll see that they\u0026rsquo;re UBlueprintGeneratedClass, being pointed to via a UClass*. The rest is working as expected though: It can find the same object we received when we check InstancesOfCDO.\nThe UBlueprintGeneratedClass type derives from UClass, and describes a UClass that was generated based on a UBlueprint. When you open the Blueprint Editor like before and add a function or a variable, a new UBlueprintGeneratedClass will be generated, containing the compiled info from your blueprint. Let\u0026rsquo;s expand that UBlueprintGeneratedClass we got:\nAt the very top, still underlined in green, you can see that the same UClass object at address 0x0000021d0d70dd00 we were looking at in the previous image. Check it out though, if you drill down to its UStruct base class, you can see underlined in red how it is pointing at UMyObject::StaticClass() as a SuperStruct (i.e. parent class). The exact same UClass is also underlined in red on the previous image.\nIf you peek a few lines below SuperStruct, you can see the Children and ChildProperties members pointing at the fields of UMyObject. Our DerivedObject doesn\u0026rsquo;t have any extra variables or functions, but if it did you would find them on the ChildProperties member owned directly by the UBlueprintGeneratedClass, underlined in light blue.\nFinally, you can see two extra things: The ClassGeneratedBy field of this UBlueprintGeneratedClass points at our UBlueprint asset we saw at the content browser. Also, we can see at the very bottom of this image how the UBlueprintGeneratedClass is pointing at the CDO of DerivedObject that we retrieved on the previous image.\nLets try modifying our DerivedObject a little bit. I\u0026rsquo;ll add an extra empty function (pointed in red), a variable, and change the CDO value for MyValue to 5.0f. You can also see the CDO\u0026rsquo;s value for the NewVariable on the right pane:\nIf we compile this, then provide a new DerivedObject to ReceiveMyObject and peek at its UBlueprintGeneratedClass again, this is what we get:\nUnderlined in blue you can see how Children now points to the new TestFunction, and how ChildProperties now points to our NewVariable. These were nullptr before. Also, Visual Studio is telling us something here: Note how our CDO at the very bottom is written in red text: This means the field has changed, and it is now pointing at a new object entirely.\nThis is because every time you compile your UBlueprint, the engine will replace all instances of your UBlueprintGeneratedClass with brand new ones (copying over any custom property values you could have), and that includes the CDO. If you want to have a look at this part of the source, I recommend starting out at this file.\nYou should know that a lot of the blueprint-related stuff is editor-only, and is not available at the cooked game. Essentially, when you compile your blueprints they all become UFUNCTIONs and UPROPERTYs, and mostly that\u0026rsquo;s all you need.\nThere\u0026rsquo;s still a couple of things I want to show you though, because they can be quite confusing at times. Check out this code:\nUClass* BaseClass = UMyObject::StaticClass(); UMyObject* BaseCDO = BaseClass-\u0026gt;GetDefaultObject\u0026lt;UMyObject\u0026gt;(); UMyObject* BaseBefore = NewObject\u0026lt;UMyObject\u0026gt;( this, BaseClass, NAME_None, RF_NoFlags, BaseCDO ); UE_LOG( LogTemp, Log, TEXT( \u0026#34;BaseBefore: %f\u0026#34; ), BaseBefore-\u0026gt;MyValue ); // This value starts out at 2.3f, but we\u0026#39;ll change it to 3.0f BaseCDO-\u0026gt;MyValue = 3.0f; UMyObject* BaseAfter = NewObject\u0026lt;UMyObject\u0026gt;( this, BaseClass, NAME_None, RF_NoFlags, BaseCDO ); UE_LOG( LogTemp, Log, TEXT( \u0026#34;BaseBefore: %f, BaseAfter: %f\u0026#34; ), BaseBefore-\u0026gt;MyValue, BaseAfter-\u0026gt;MyValue ); And the analogue for DerivedObject:\nUClass* DerivedClass = LoadClass\u0026lt;UMyObject\u0026gt;( NULL, TEXT( \u0026#34;Blueprint\u0026#39;/Game/DerivedObject.DerivedObject_C\u0026#39;\u0026#34; ) ); UMyObject* DerivedCDO = DerivedClass-\u0026gt;GetDefaultObject\u0026lt;UMyObject\u0026gt;(); UMyObject* DerivedBefore = NewObject\u0026lt;UMyObject\u0026gt;( this, DerivedClass, NAME_None, RF_NoFlags, DerivedCDO ); UE_LOG( LogTemp, Log, TEXT( \u0026#34;DerivedBefore: %f\u0026#34; ), DerivedBefore-\u0026gt;MyValue ); // This value starts out at 5.0f, but we\u0026#39;ll change it to 6.0f DerivedCDO-\u0026gt;MyValue = 6.0f; UMyObject* DerivedAfter = NewObject\u0026lt;UMyObject\u0026gt;( this, DerivedClass, NAME_None, RF_NoFlags, DerivedCDO ); UE_LOG( LogTemp, Log, TEXT( \u0026#34;DerivedBefore: %f, DerivedAfter: %f\u0026#34; ), DerivedBefore-\u0026gt;MyValue, DerivedAfter-\u0026gt;MyValue ); A couple of things are worth mentioning before we analyze the output:\nYou can load your blueprint class using that LoadClass call, but also don\u0026rsquo;t miss the fact that we\u0026rsquo;re providing the path \u0026ldquo;Blueprint\u0026rsquo;/Game/DerivedObject.DerivedObject**_C**\u0026rsquo;\u0026rdquo;. The _C suffix forms the path we need to retrieve the UBlueprintGeneratedClass stored inside the UBlueprint asset; To create our DerivedObject instances we\u0026rsquo;re calling NewObject\u0026lt;UMyObject\u0026gt; instead. We can\u0026rsquo;t just do NewObject\u0026lt;DerivedObject\u0026gt; as DerivedObject is not a C++ type, it\u0026rsquo;s just a blueprint class. This is perfectly fine though: We\u0026rsquo;re providing DerivedClass to NewObject anyway. It will create an instance of the derived class (a DerivedObject), and return a polymorphic pointer to it. Anyway, when we run his code, we get this output:\nLogTemp: BaseBefore: 2.300000 LogTemp: BaseBefore: 2.300000, BaseAfter: 2.300000 LogTemp: DerivedBefore: 5.000000 LogTemp: DerivedBefore: 5.000000, DerivedAfter: 6.000000 Notice how updating the value in the CDO had no effect on new UMyObject instances, while it affected new DerivedObject instances. As it turns out, in the general case only blueprint classes (i.e. types whose class is a UBlueprintGeneratedClass) automatically get the provided template\u0026rsquo;s values upon construction (you can have a look at where this is checked for over here). This can be confusing because the NewObject function has a doc comment that suggests it would always copy things over from the CDO, but apparently not.\nAlso notice how in neither case the objects constructed before we changed the CDO were automatically updated to the new values when we modified the CDO. This can also be a bit confusing because the \u0026ldquo;automatic update\u0026rdquo; behavior only happens if the change to the CDO is done via the Editor. As far as I can tell there\u0026rsquo;s no easy way of doing this via C++, because the code is private to the PropertyEditor module, but if you need that functionality you can have a look at how the Property Editor does it and replicate it.\nAs a closing remark, note that I edited the CDO like this to prove a point, but be careful when doing that in your project: The engine only creates one CDO for each class, and that is used as template for instances spawned in the editor, as well any instances spawned when you\u0026rsquo;re testing via Play In Editor. Additionally, the CDO\u0026rsquo;s values for UBlueprintGeneratedClasses are saved directly to the UBlueprint asset. This means that if you go into Play In Editor, get your CDO modified, then exit Play in Editor and save the UBlueprint asset, those changes would persist forever, and this is likely not what you want.\nConclusion Congratulations on surviving this UE4 whirlwind tour!\nThis turned out a lot larger than I thought it would, sorry about that. Even so there are many more things to talk about, but hopefully this helps to get some traction with the base UE4 C++ classes. It is quite a lot of stuff to take in at once, that\u0026rsquo;s for sure.\nI have some next posts already planned, but leave a comment if there\u0026rsquo;s anything in particular you want me to talk about or explore next, or if you find out any mistakes or badly explained sections. I\u0026rsquo;ll likely refer to this post in the future, so it would be neat if it was kept in tip-top shape.\nThanks for reading!\n","permalink":"https://1danielcoelho.github.io/unreal-engine-basics-base-classes/","summary":"\u003cp\u003eThis is going to be the first of a few of blog posts detailing some stuff I\u0026rsquo;ve learned doing deep dives in the Unreal Engine source.\u003c/p\u003e\n\u003cp\u003eThis is mostly to get the ball rolling as far as this blog is concerned (as working with Unreal is a large part of my day job and should be the easiest to write about), but also because UE\u0026rsquo;s documentation is not the best, and there\u0026rsquo;s a surprising shortage of posts like this out there. It seems like the community just assumes everyone is forced to dig through the code, and while there\u0026rsquo;s nothing fundamentally wrong with reading code, it does get in the way a bit when all you want is an overview. It also sucks that everyone has to rediscover the same insights over and over, so hopefully this can provide a net saving of man hours to the world.\u003c/p\u003e","title":"Unreal Engine basics and base classes"},{"content":"This is my first post on this blog.\nHopefully there will be others\u0026hellip;\n","permalink":"https://1danielcoelho.github.io/first/","summary":"\u003cp\u003eThis is my first post on this blog.\u003c/p\u003e\n\u003cp\u003eHopefully there will be others\u0026hellip;\u003c/p\u003e","title":"First post"}]