CGReminders by SonicTK

I've written some words about working with x64 assembly on Windows. If you're interested in learning programming fundamentals, I welcome your feedback on my attempt at instruction.

It’s been a while since my last tutorial, and the reason is because I’ve been busy writing another 5088 lines of another tutorial, this time expanding on a presentation I gave internally at Blizzard during our last hackathon: how to work with assembly on Windows, and an overview of how a knowledge of assembly can be useful.

Unlike my other tutorials, this will notbe a tutorial for the…

It’s been a while since my last tutorial, and the reason is because I’ve been busy writing another 5088 lines of another tutorial, this time expanding on a presentation I gave internally at Blizzard during our last hackathon: how to work with assembly on Windows, and an overview of how a knowledge of assembly can be useful.

Unlike my other tutorials, this will not be a tutorial for the absolute beginner who’s never touched a line of code before in their lives; you are expected to have some pre-requsite knowledge of C/C++ here.

Happy reading! And as always, if you notice errors or have questions, don’t hesitate to post here or reach me on Twitter.

Writing a Maya Python C++ Extension

I’ve written my first tutorial for 2018. It’s been a while, since I’ve been busy learning some other things and getting caught up with other non-programming related issues.

This time, it’s inspired by a topic that has been brought up several times by several acquaintances of mine; the relative lack of completeness surrounding the Maya Python API 2.0. So, instead of endlessly hearing complaints…

I’ve written my first tutorial for 2018. It’s been a while, since I’ve been busy learning some other things and getting caught up with other non-programming related issues.

This time, it’s inspired by a topic that has been brought up several times by several acquaintances of mine; the relative lack of completeness surrounding the Maya Python API 2.0. So, instead of endlessly hearing complaints about it, I’d like to present people with the tools they need to get the work done. By themselves, if necessary.

Here’s how you write Python C++ extensions that can be used in Maya.

[vc_row][vc_column][vc_column_text]It seems like everyone’s up to getting hot-reloadable code working in their custom engines these days, so I thought I’d take a stab at my own implementation inside a Maya deformer (though it doesn’t have to be limited to just a deformer!) This is basically a technique that allows you to dynamically reload compiled code on-the-fly, all without having to unload/reload your plugin or worse, restart Maya.[/vc_column_text][vc_video link=”https://youtu.be/exGlnEJGmlI” align=”center” title=”Runtime Compiled C++ in Maya”][vc_column_text]Continuing in the vein of my previous tutorial, I thought I’d try using the same approach of generating a tutorial outside of the scope of a normal basic blog post, which I got some good feedback about. The unfortunate side effect is that this has quickly become the longest tutorial I’ve ever written thus far, and yet it’s still incomplete in a multitude of areas. (Perhaps I’ll update it in the future to address some of its knowledge gaps)

You can take a look at the tutorial here, and view the Github repository that contains the sample code here.[/vc_column_text][/vc_column][/vc_row]

[vc_row][vc_column][vc_column_text]Raffaele Fragapane has done a great job with his Cult of Rig series with getting more people interested in low-level rigging and getting back to CS fundamentals when it comes to designing rigs. Something I found interesting in the way he did his rig features was using a script node to handle installation of callbacks. Script nodes, however, present several significant issues when it comes to production.

[/vc_column_text][vc_column_text]Instead of rambling on about it here, however, for once, I’m going to try something a little different; due to the length of this tutorial, I’m hosting it on Github instead, along with the example source code.

You can go ahead and read my thoughts on how I’d approach the same problem he tries to solve here.[/vc_column_text][/vc_column][/vc_row]

Docset for 3ds max C++ SDK

[vc_row][vc_column][vc_column_text]In the same spirit of the Maya one, I’ve tweaked my scripts a little to generate a Dash-compatible docset for the 3ds max C++ SDK reference as well. Yes, catering to niche needs is indeed my manifesto.[/vc_column_text][vc_video link=”https://youtu.be/R0KUOjIcsNo” align=”center”][vc_column_text]You can get the code used for generating the docset via my…

[vc_row][vc_column][vc_column_text]In the same spirit of the Maya one, I’ve tweaked my scripts a little to generate a Dash-compatible docset for the 3ds max C++ SDK reference as well. Yes, catering to niche needs is indeed my manifesto.[/vc_column_text][vc_video link=”https://youtu.be/R0KUOjIcsNo” align=”center”][vc_column_text]You can get the code used for generating the docset via my Github.

Unfortunately, unlike the Maya one, I’ve elected not to include a full, ready-to-use version of the docset in this one due to the sheer size of the 3ds max documentation; the sources alone eclipse both 2016 and 2017 versions of the Maya docsets combined in terms of size. You’ll need to download the offline developer documentation for 3ds max from Autodesk’s official documentation website. Instructions for generating the docset are available in the repository’s README.md file.

Hopefully this helps out folks who are still stuck supporting 3ds max; I know firsthand how painful and slow searching through the official documentation can be. If you notice any bugs while using this, feel free to let me know and/or submit a pull request![/vc_column_text][/vc_column][/vc_row]

[vc_row][vc_column][vc_column_text]Something that still shocks me this day is just how many developers out there write code without the aid of a debugger or profiler. I understand that not everyone has the time/energy to test every single aspect of their written applications, but in my mind, that’s just setting yourself up for failure. How can you possibly be sure that your code is as performant as possible without stepping through it to see what the stack/heap is every step of the way? Relying on coverage tests or unit tests alone is also not a definitive way to ensure that your code is performing as you originally intended, especially when it comes to topics such as recursion. And what about trying to diagnose problems with 3rd-party libraries, whose implementation you may not be as familiar with?

But regardless of my view on aggressive debugging during development, I’ve come to the realization that most people don’t debug their code simply because they don’t know how to do it. Oh sure, they might have set it up at one point, but then their configuration changed, or they had to re-install PyCharm, or “Autodesk changed something” and now their setup simply doesn’t work anymore, etc.

Today, I’m going to try and provide clarity on just what exactly you need to do to be able to debug your compiled C/C++ plugins in Maya, for all 3 major platforms.[/vc_column_text][vc_tta_accordion active_section=”1″ collapsible_all=”true”][vc_tta_section i_icon_fontawesome=”fa fa-file-code-o” title=”Getting the example code” tab_id=”1479700380025-21361231-ef82″ add_icon=”true”][vc_column_text]The first step, of course, is that we’ll need some code to actually step through. I’m using CMake for my project builds in Maya, so the first step is to setup the project builds across the major platforms.

I’ve uploaded the sample project code to Github, which is as barebones as it gets.

I use a slightly modified version of Chad Vernon’s FindMaya.cmake module, which does the brunt of the work for setting up Maya plugin builds. For this tutorial, we’re going to be compiling an example MPxGeometryFilter from the Maya devkit, namely, the identityGeomFilter.cpp file provided. You can find it yourself in devkit/plugins/identityGeomFilter.

The plug-in itself is fairly simple; it does absolutely nothing at all apart from setting the positions of the input geometry to that of the original mesh’s positions, which makes it perfect for testing.

I’m going to make the following assumptions here:

That you know a little about C/C++  and how to use it to make Maya plug-ins (at least, enough to have already compiled your plug-in before and run it in action)

That you know how to use your IDE that you’re comfortable in enough to be able to configure it up the wazoo to suit the instructions given

That you won’t start talking about SCons instead in the comments, because seriously, I don’t want to hear it. (Only half-joking here)

[/vc_column_text][/vc_tta_section][vc_tta_section i_icon_fontawesome=”fa fa-windows” title=”Windows” tab_id=”1479007462493-1f9f5931-5df7″ add_icon=”true”][vc_column_text]For Windows, things aren’t as complicated as they seem…at least, at a first glance. You’ll need the following dependencies installed for Maya 2016 Extension 2:

Visual Studio 2012 Update 4 + Win 8 SDK

CMake (>2.8)

Personally, I run with VS 2015 as my debugging IDE and have older versions of Visual Studio installed for compiling for older versions of Maya as necessary. You could run a newer version of the compiler to compile your plugins, but there’s no telling when or what might break in specific situations (though I can’t say I’ve personally ever experienced anything of the sort yet, why tempt fate?)

To compile my plug-ins, I use CMake. To generate the build files, first create a directory called build in the root directory and then navigate to it. Once inside, run the following commands:

cmake -G "Visual Studio 11 2012 Win64" -DMAYA_VERSION=2016 -DCMAKE_BUILD_TYPE=Debug ../

This will generate the Visual Studio solution for the project, which can then be opened in a new Visual Studio session. You can then either build the plugin within Visual Studio itself or run the command:

cmake --build . --config Debug --target install

Once that’s done, you’re ready to debug! Open up the Visual Studio solution first. You now have two options for debugging…

The quick way (attaching to the Maya process) is as follows:

Launch Maya.

Run the following command:

tasklist /fi "IMAGHENAME eq maya.exe"

You should see something like this as your output

[/vc_column_text][vc_single_image image=”875″ img_size=”full” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Remember the Process ID listed there. You’re going to need it.

Back in Visual Studio, click on Debug > Attach to Process. You should see a window pop up. Make sure your settings are the same as shown, and then click Attach. (You can use the PID that you retrieved previously to search for the process more easily)

[/vc_column_text][vc_single_image image=”876″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Go ahead and set a breakpoint at line 93 of the source file.

[/vc_column_text][vc_single_image image=”878″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Now load your plugin in Maya, either through bog-standard loadPlugin or whatever fancy deployment system you have. Just make sure it’s the same .mll that you actually built…

You should now be able to hit your breakpoint! Just create the deformer on a cube or something and watch the magic happen!

[/vc_column_text][vc_single_image image=”879″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]The other method, which requires a bit more setup, but is way more flexible, involves setting up Maya as your debug executable. This allows you to attach the debugger to the process as it launches, essentially putting it under the control of the debugger. This way, you can do things such as modifying the environment that it launches in, pass custom flags to the Maya command being run, etc. All very useful.

First, set your exampleDebugProject as the Startup Project. This is so you don’t get an error when trying to launch the debug configuration in Visual Studio.

[/vc_column_text][vc_single_image image=”880″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Next, you want to tell Visual Studio the location of the executable to launch under debugger control. Right-click your project, choose Properties, and then go to Debugging. Change the value of Command to be the path to your Maya executable, as shown:

[/vc_column_text][vc_single_image image=”881″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Hit OK and now hit F5 (Or click Start Debugging). Maya should now launch under the control of the debugger. Now go ahead, set your breakpoints and hit them!

[/vc_column_text][/vc_tta_section][vc_tta_section i_icon_fontawesome=”fa fa-apple” title=”OSX” tab_id=”1479007462575-b65a5b33-e4df” add_icon=”true”][vc_column_text]Just like Windows, things appear fine at first glance when you’re on OSX. You’ll need the following dependencies installed for Maya 2016 Extension 2:

XCode 6.1.1 with SDK 10.9 (Mavericks)

clang with libc++

Personally, I run with XCode 7.1 as my debugging IDE (because the App Store hangs every time I try to update to 8.0, thanks Apple!)

To generate the build files using CMake on OSX, first create a directory called build in the root directory and then navigate to it. Once inside, run the following commands:

cmake -G "Xcode" -DMAYA_VERSION=2016 -DCMAKE_BUILD_TYPE=Debug ../

This will generate the XCode project, which can then be opened in a new XCode session. You can then either build the plugin within XCode itself or run the command:

cmake --build . --config Debug --target install

Once that’s done, you’re ready to debug! Open up the XCode project first. You now have two options for debugging…

The quick/slightly unreliable way (attaching to the Maya process) is as follows:

Launch Maya.

Run the following command:

ps aux | grep -i maya

You should see something like this as your output

sonictk          1704   0.0  4.0  6231112 667236   ??  SX    7:41PM   0:37.23 /Applications/Autodesk/maya2016/Maya.app/Contents/MacOS/Maya

Remember the Process ID listed there. You’re going to need it.

Back in XCode, click on Debug > Attach to Process > Attach to Process by PID or Name. Enter the PID of the Maya process that you found previously. Alternatively, you can also use the Debug > Attach To Process > Maya route directly, though I’ve found that sometimes to not be as reliable for whatever reason.

Go ahead and set a breakpoint at line 93 of the source file in XCode.

[/vc_column_text][vc_single_image image=”886″ img_size=”large” add_caption=”yes” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

Now load your plugin in Maya, either through bog-standard loadPlugin or whatever fancy deployment system you have. Just make sure it’s the same .bundle that you actually built…

You should now be able to hit your breakpoint! Just create the deformer on a cube or something and watch the magic happen!

…Wait, you weren’t able to hit your breakpoint? Welcome to the random shit that happens on OSX.

The other method, which requires a bit more setup, but is way more flexible and reliable on OSX, involves setting up Maya as your debug executable. This allows you to attach the debugger to the process as it launches, essentially putting it under the control of the debugger. This way, you can do things such as modifying the environment that it launches in, pass custom flags to the Maya command being run, etc. As a matter of fact, this is the most hassle-free way on OSX to launch Maya under a custom environment.

First, let’s set up our XCode project so that it launches Maya as well under the control of LLDB (XCode’s debugger) when we fire off a build. Go to Product > Scheme > Edit Scheme:

  [/vc_column_text][vc_single_image image=”887″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]

The next fun part is actually setting the debug “executable”. Because setting it to just Maya.app as you normally would for a OSX application results in Maya just printing out worthless help information, we’re going to have to open a new Finder window, and navigate to the Maya.app itself, right-click it, choose Show Package Contents, then navigate to Contents/bin/maya, and then drag-and-drop that to the Executable window (Make sure you choose other for the Executable first to bring up the file chooser dialog).

Now you should be able to hit your breakpoint.

[/vc_column_text][vc_single_image image=”888″ alignment=”center” style=”vc_box_rounded” onclick=”link_image”][/vc_tta_section][vc_tta_section i_icon_fontawesome=”fa fa-linux” title=”Linux” tab_id=”1479007591257-49c46236-1e7d” add_icon=”true”][vc_column_text](For Linux, my development environment has been tested in Fedora 23, CentOS7 and EL6.5 environments. I don’t roll with Debian-based distributions mostly because RHEL is the officially-supported platform for a lot of DCCs. However, that being said, there’s no reason the following workflow shouldn’t work on any popular Debian-based distro that you might be using)

For Linux, it’s probably the easiest of the 3 to setup. All you really need to do is run Maya under the control of the debugger you’re trying to use (which is probably gdb or ddd). You can do so with the simple command:

maya -d gdb (or ddd)

But obviously, this is very sad and puerile. So we’re going to do things my way. That is, with Emacs! (Or whatever the hell you want, really. Just make sure it has a gdb front-end or something. CLion will work as well.)

For this, you’re going to need the following dependencies:

GCC 4.8.2

To compile my plug-ins, I use CMake. To generate the build files, first create a directory called build in the root directory and then navigate to it. Once inside, run the following commands:

cmake -G "Unix Makefiles" -DMAYA_VERSION=2016 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS_DEBUG="-Wall -ggdb" ../

This will generate the Makefile for the project, which can then be either built with Make itself or you can run the command:

cmake --build . --config Debug --target install

Once that’s done, you’re ready to debug! Open up your favorite IDE of choice which has a gdb front-end.

Launch Maya.

Run the following command:

ps aux | grep -i maya.bin

You should see something like this as your output

sonictk  13696  0.0  0.0 125016  2876 tty2     S+   23:27   0:00 /bin/csh -f /usr/autodesk/maya/bin/maya sonictk  13709  7.6  3.6 44450148 898036 tty2  Sl+  23:27   0:21 /usr/autodesk/maya/bin/maya.bin

You’re looking for PID 13709, in this case. (Yes, I run ZShell. Get over it.)

Run the following command in your IDE to attach to the Maya process:

gdb attach <pid>

(For Emacs, add the -i=mi flag as well in order for gud-gdb to activate)

Once there, set a breakpoint at the usual place:

[/vc_column_text][vc_single_image image=”890″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][vc_column_text]And that’s it! You should be able to hit your breakpoint now once you reach it in Maya:[/vc_column_text][vc_single_image image=”891″ img_size=”large” alignment=”center” style=”vc_box_rounded” onclick=”link_image”][/vc_tta_section][/vc_tta_accordion][/vc_column][/vc_row][vc_row][vc_column][vc_column_text]So there you have it. A little bit of effort, but now you have the ability to debug infinitely more complex problems, especially tricky ones that happen sporadically as well in deform()/compute() functions!

No more excuses![/vc_column_text][/vc_column][/vc_row]

A colleague of mine came across an oddity when working with the Maya Python API the other day, which serves as a poignant reminder that again, no matter what language we’re working with, or however many layers of abstractions there are between us, ultimately, we are dealing with memory, and we would do well to remember as such.

Virtual environments are pretty well-established in Python, and have been so for quite some time; however, after chatting with a couple of TD acquaintances, and even seasoned programmers in the game industry, I’m still running into people who either have never heard of them(!), or don’t use them because “they’re too troublesome to setup”, or…

Continue reading "Start using virtual environments! (instead of mayapy)"…

So as I’m starting to expand my knowledge of Maya and slowly weaning myself off the tragedy of both PyMEL (and cmds, yes, I do use it sometimes /noshame), OpenMaya  and onto more C++ and the Python API 2.0 in general, I’m going to make an admission here: I’m not the kind of guy who…

Continue reading "Python completion files for OpenMaya 2 and Mari"…

Well, this is it, the long-procrastinated post. I’ve been meaning to write this for a while now; a common problem…

Continue reading "Deploying applications with Inno Setup"…

I’ve been running into this issue a lot recently when dealing with virtualenvs with my Python projects. (Which forms the majority of my work, so it’s definitely not just a minor concern.) I’m planning on writing a post on why working with virtualenvs for me is so important these days, especially when dealing with larger projects that could span multiple working environments, and especially more so when you need to collaborate with others as well. But that’s outside the scope of this post for now.

Essentially, as shown above, sometimes PyCharm will, for whatever reason, decide that the best solution for you adding a new Python interpreter is to add it as a duplicate entry. Removing any one of the entries then results in all of them being removed. Re-generating the caches, removing the project and re-adding it, un-registering it as a VCS root…every action via the GUI as far as I can tell is unable to resolve the issue.

To solve this, go to your PyCharm user configuration folder in $HOME/.Pycharm40 and open the jdk.table.xml file located there. (e.g. C:\Users\sonictk\.PyCharm40\config\options\jdk.table.xml)

You should be able to find instances of the duplicate entries and remove them that way. (Between each <jdk> element)

Hopefully this helps out anyone who’s been running into the same issue! I know I’ve talked to a few people before who have resigned themselves to ignoring it when it happens. :)

I’ve made a new syntax highlighter for ZBrush’s ZScript scripting language in Sublime Text, which you can grab from Package Control. You can also just install it from within Sublime Text through the Command Palette as well.

It supports syntax highlighting and auto-completion of all commands as per the ZBrush documentation.

Hopefully this helps out any fellow artists/sculptors who might be looking to get into a little bit of ZBrush automation!

(And if you’re a fellow developer who would like direct access to the source, you can always visit the source BitBucket repo.)

Yesterday, when I was trying to run my copy of 3ds max and Maya, I was suddenly presented with the following cryptic exception message:

An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information.

Wait, what the hell? So obviously I assumed it had something to do with the way I had recently changed some stuff I was working on, whether it was to do with win32 COM or screwing around with python subprocesses. After spending two hours getting nowhere re-installing MFC redistributables and .NET runtimes and finding not much information on this particular error, turns out it was due to Mercurial, of all things.

This error can possibly occur when you have an application that adds a incorrectly-compiled version of msvcr90.dll to the system PATH...which in this case was Mercurial. Removed it from PATH, and all was well.