#pygments.rb

Snipplets, the syntax highlighting app I built a month ago, makes strong use of Pygmentizer to highlight user submitted code segments. It does so by making a POST request to an external pygmentizer app located on google app engines which then makes a successful request back to the app on completion of the highlighting.

While this serves the purpose of the app, it has some drawbacks. Snipplets relies on a Resque queue to wait for the request to be complete which means the user ends up with an inconsistent view of the codesnipplet; also what happens if the API service fails?

After watching the railscasts episode 207 which introduces pygments.rb I ported all the syntax highlighting code to using pygments.rb.

Whats kool about pygments.rb is that it still uses pygemntizer but the program itself is embedded within the gem and your app using ruby-python and libffi ( through the ffi gem ) which means any environment that supports or runs python can run pygments.rb and that includes Heroku.

However I ran into this peculiar bug whereby pygments.rb crashes the entire application server ( both Mongrel and Webrick as shown in the Railscasts) with a killed signal message on the console when running Pygments.highlight method.

If you are running into the same issue, try switching over to Thin which works for me without any issues. As yet I still cannot ascertain the error whether it is system dependent or not and ruby-debug did not thrown up any unusual bugs. Could have something to do with asynchronous processing but will look into it in a separate post.

If you are new to using pygments.rb and have just switched over you might notice that the line numbers have disappeared completely if you used to have line numbers enabled. This is off by default in the gem and can be enabled by passing the line numbers options to true inside the Pygments.highlight method.

Below is a short snipplet describing some of the workarounds for pygments.rb as discussed above:

The second issue you might run into when deploying onto Heroku is the version of RubyPython you are using. If you get errors such as 'lexer cannot be found' then it is related to RubyPython as pygments.rb cannot pick up on the Python interpreter. To do so simply create an initializer file ( I called mine 'rubypython.rb' and put it inside config/initializers) with the following code:

Essentially what the above does is to start a python interpreter session so that pygmentizer can run. I can't seem to find any other way to avoid this else the entire app will fail. The other gotcha is that it only seems to work with RubyPython 0.5.1 so I have the gemfiles locked down to that version. I tried it with both RubyPython 0.5.3 and the latest 0.6.1 to no avail - the cedar stack on heroku just refuses to communicate with rubypython.