Pages

Tuesday, May 13, 2014

Blender - FBX Support for Custom Vertex Normals

NOTE: These instructions are pretty outdated but should still work on older versions of Blender using the FBX 6.1 ASCII exporter. Newer versions of Blender can export custom normals without these edits.
I've also been working on a normal editor for Blender 2.74+ that is available here (Github)

Old Post:
A large drawback to using Blender for game art is that, by default, it has no method for modifying vertex normals, making it hard to get proper shading on foliage meshes consisting of planes, such as grass. A while back, a user named adsn on BlenderArtists.org released an addon that allows Blender to recalculate vertex normals based on the 3D cursor's location (or manually by entering rotation vectors per vertex) that had one minor issue: Blender recalculates normals when you export a file, basically erasing the work this plugin does on meshes.

After some modifications to an older version of the fbx exporter, it seems I've figured out how to get Blender to export custom normals generated by the plugin mentioned above.


Note:  I had a message here about not being able to export multiple objects, but apparently that was incorrect. As long as the custom normals have been saved on an object once, they will be exported. (collision meshes can also be exported) After some experimentation, it turns out that sharp edges are also no problem.


UDK: This method will only work on static meshes, since UDK recalculates normals on import for skeletal meshes.

Meshes exported with this should be compatible with every application that allows you to import FBX meshes with custom vertex normals.


Requirements: 

- asdn's Recalc Vertex Normals Addon from BlenderArtists.org (linked thread includes usage instructions)
- The fbx export script from Blender 2.68 or earlier
- Blender (any recent version)
- a text editor (preferably something with line number display)


Preparation:

- Get Blender 2.68 or earlier. The 7z archive would be best option so you can just extract the required script without reinstalling.

- Find the folder containing the fbx exporter script in this archive ('io_scene_fbx' in Blender\(version no.)\scripts\addons\ ), copy it somewhere, rename the copy to anything else and place it in your installation's addons directory (Blender\(version no.)\scripts\addons\ ).

- Open __init__.py in this new folder and change the plugin name ("name": at around line 22) so you can find it in the plugin manager.

- Go to around line 272 to change the text that is displayed in the export menu:


def menu_func(self, context):
    self.layout.operator(ExportFBX.bl_idname, text="Autodesk FBX (.fbx)")



Change it to something like text="Autodesk FBX w Normals(.fbx)" so you know which exporter script you're using.


*Note: Since there's no official support for the modded plugin, you may want to remove the wiki url and change the support type to something other than 'Official', too, but this has nothing to do with the functionality.


Script Changes:

Since asdn's addon stores its list of vertex normals in the active object, all you need to do is copy that list and replace the fbx exporter's file write operations that cover normals.

- Open export_fbx.py
The actual file writing starts at around line 1352. The easy way to get there would be to search for the function definition, "def write_mesh(my_mesh):". You should see something like this:


def write_mesh(my_mesh):

        me = my_mesh.blenData

        # if there are non NULL materials on this mesh
        do_materials = bool(my_mesh.blenMaterials)
        do_textures = bool(my_mesh.blenTextures)
        do_uvs = bool(me.tessface_uv_textures)
        do_shapekeys = (my_mesh.blenObject.type == 'MESH' and
                        my_mesh.blenObject.data.shape_keys and
                        len(my_mesh.blenObject.data.vertices) == len(me.vertices))

        fw('\n\tModel: "Model::%s", "Mesh" {' % my_mesh.fbxName)
        fw('\n\t\tVersion: 232')  # newline is added in write_object_props

        # convert into lists once.
        me_vertices = me.vertices[:]
        me_edges = me.edges[:] if use_mesh_edges else ()
        me_faces = me.tessfaces[:]



- Add the following below this:


new_normals_list = bpy.context.active_object.vertex_normal_list[:]


Note: Make sure you use spaces for the indentation since mixing tabs and spaces will result in error messages.


Now find the actual file write operation for the normals, which should be at around line 1469. It looks like this:


        i = -1
        for v in me_vertices:
            if i == -1:
                fw('%.15f,%.15f,%.15f' % v.normal[:])
                i = 0
            else:
                if i == 2:
                    fw('\n\t\t\t ')
                    i = 0
                fw(',%.15f,%.15f,%.15f' % v.normal[:])
            i += 1
        fw('\n\t\t}')



- Replace me_vertices with new_normals_list.

That's it. Now open Blender, find the addon in the addon manager and enable it. The renamed FBX export option should now export the custom normals generated by asdn's script


When importing into UDK, make sure that the Import Tangents and Explicit Normals options are checked.


I'm aware that it's not an especially clean solution since I'm not checking if the variable exists, so the script will fail if you don't use asdn's script before export. In its current state, it's a quick and dirty fix/hack.

I'm looking into getting this method to work with the new version of the exporter, and may end up releasing it as a full export script depending on what I find out about permissions. (I know it's licensed under GNU, but I like to get permission before releasing modified versions of other people's work anyways).


Screenshots:

I don't have any decent tree meshes to show it on at the moment but here are some examples of grass sheets, a sphere and a cube to show the effect in UDK. All of these are taken in the static mesh editor inside UDK for now.


The first two are Blender's automatically calculated normals on the left and the typical way grass planes are done (normals pointing up) on the right. There's a huge difference in the way light is interacting with the surfaces.



The next three images are a few different vertex normal layouts I was playing around with. I imported the cube and sphere meshes at a very small scale to make the lines representing the normals more obvious in the screenshots.





5 comments:

  1. I downloaded your addon from https://github.com/isathar/Blender_UDK_FBX-Exporter/commits/master and I reverted to version 2.68 to export some of my work to FBX.

    I get the following error...

    FBX export starting... 'C:\\Users\\James\\Desktop\\Country Road\\LandscapeTest1.
    fbx'
    Calculating Tangents and Binormals...
    Traceback (most recent call last):
    File "C:\Users\James\Desktop\blender-2.68-windows64\2.68\scripts\addons\io_sce
    ne_fbxcust\__init__.py", line 226, in execute
    return export_fbx.save(self, context, **keywords)
    File "C:\Users\James\Desktop\blender-2.68-windows64\2.68\scripts\addons\io_sce
    ne_fbxcust\export_fbx.py", line 3157, in save
    return save_single(operator, context.scene, filepath, **kwargs_mod)
    File "C:\Users\James\Desktop\blender-2.68-windows64\2.68\scripts\addons\io_sce
    ne_fbxcust\export_fbx.py", line 2517, in save_single
    write_mesh(my_mesh)
    File "C:\Users\James\Desktop\blender-2.68-windows64\2.68\scripts\addons\io_sce
    ne_fbxcust\export_fbx.py", line 1457, in write_mesh
    temp_smoothgroups = bpy.context.active_object.fbx_smoothing_groups[:]
    AttributeError: 'Object' object has no attribute 'fbx_smoothing_groups'

    location: :-1

    location: :-1

    Any ideas on the fix please? Thanks

    ReplyDelete
    Replies
    1. Hey, thanks for reporting this. I just realized I never updated the exporter-only version of the scripts after the initial upload. The error is my fault, and I should have it fixed and updated within the hour.
      Until then, the exporter in my fbx tools addon is more recent and shouldn't have this problem.

      Delete
    2. A fixed and updated version of the exporter should be on Github now.

      Delete
    3. Also, you shouldn't need to revert to a previous Blender version; it's working on the most recent build (2.72).

      Delete
  2. That's excellent, thanks so much for the quick fix, it works perfectly and with asdn's addon, the normals are looking the way they should.

    One quick note, the export took about 2m 30s on my i7 4820K, I suspect this is something to do with the extra calculations required, but you may want to put a note in the readme in case some people assume blender has crashed.

    Thanks again.

    James

    ReplyDelete