Clang AST parsing for automated code generation

本文主要是介绍Clang AST parsing for automated code generation，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

原文地址：http://www.seethroughskin.com/blog/?p=2172

Syntax traversal is a powerful tool. With it you can automate repetitive tasks, search for semantic errors, generate wrappers, and so much more.  A few months ago I hit a hump (read: a f***ing mountain) of an issue with some legacy code that has been on my plate for awhile now.

Having killed a small forest’s worth of paper I decided that manually tracing paths through code was an inefficient use of my time.  Instead I went in search an automatic method for generating an abstract syntax tree(AST) for C++ code.  My idea was that I could use the AST to generate something like a direct graph to better visualize code flow.

There are a few flavors of readable syntax generation out there (and likely more):

• pycparser (Supports on C, I believe)
• gccxml
• Clang (via AST dump)

I’ve been a fan of Clang for awhile now and they have a very robust and active community making it a natural choice for my AST generation needs.  Clang also has decent articles on getting started in both Windows and Linux.  If you don’t have Clang installed, I suggest reading that linked article.  You’ll need compiled versions of clang.exe and libclang.dll to follow along with the Python binding below.

[Caveat]

Clang at revision 183352 (2013-06-05)  has a slight issue in that it won’t identify Linkage specifications (e.g. extern “C” void foo()).  To fix this issue, follow these steps from my SO answer:

//Bit of a necroanswer but if you go in to \llvm\tools\clang\lib\Sema\SemaCodeComplete.cpp and add the following line:case Decl::LinkageSpec:  return CXCursor_LinkageSpec;//To the switch in:
CXCursorKind clang::getCursorKindForDecl(const Decl *D)//It should resolve the issue of clang's Python binder 
//returning UNEXPOSED_DECL instead of the correct LINKAGE_SPEC.
//This change was made at revision 183352(2013-06-05).//Example from my version:
CXCursorKind clang::getCursorKindForDecl(const Decl *D) {
if (!D)return CXCursor_UnexposedDecl;switch (D->getKind()) {case Decl::Enum:               return CXCursor_EnumDecl; case Decl::LinkageSpec:  return CXCursor_LinkageSpec;// ......

[Libclang]

Libclang is Clang’s dynamic binding that is used in conjunction w/ Python to allow for interpreted code evaluation.  Eli Bendersky has a great post on using libclang that I referenced frequently while writing code.  Clang documentation can be very lacking in some areas and Eli’s post does a good job of explaining the steps to getting libclang working with Python.  If you follow his steps the basic pipeline is:

• Compile libclang
• Add libclang to your PATH environment variable
  • On *Nix it’s LD_LIBRARY_PATH
  • On Windows it’s the standard PATH
  • Or do it in python: os.environ['PATH'] = ‘/path/to/libclang’
• Copy the Clang/Python bindings from /llvm/tools/clang/bindings/python to your python installation or however you’d prefer to install it.
• Verify it works by opening a python console and typing: improt clang.cindex
• Squee when it works

[Example]

Once libclang is tied to Python it’s time to test your code.  When I got to this step I had trouble finding any good examples.  There are really only 2 and they can be found in your Clang installation folder: llvm\tools\clang\bindings\python\examples\cindex.  Others can be gleaned from blog posts and StackOverflow.  Here is a simple example I adapted that looks specifically for the LINKAGE_SPEC cursor type. LINKAGE_SPEC refers to code like `extern “C”`

#!/usr/bin/env pythonimport os
import sys
from pprint import pprint
import clang.cindex
os.environ['PATH'] = os.environ['PATH']  + os.getcwd()def get_info(node, depth=0):return { 'kind' : node.kind,'usr' : node.get_usr(),'spelling' : node.spelling,'location' : node.location,'extent.start' : node.extent.start,'extent.end' : node.extent.end,'is_definition' : node.is_definition()}def output_cursor_and_children(cursor, level=0):#LINKAGE_SPEC (http://clang.llvm.org/doxygen/classclang_1_1LinkageSpecDecl.html)#Represents code of the type:  extern "C" void foo()if cursor.kind == clang.cindex.CursorKind.LINKAGE_SPEC:pprint(('nodes', get_info(cursor)))# Recurse for children of this cursorhas_children = False;for c in cursor.get_children():if not has_children:has_children = Trueoutput_cursor_and_children(c, level+1)def main():from clang.cindex import Indexfrom pprint import pprintfrom optparse import OptionParser, OptionGroupglobal optsparser = OptionParser("usage: %prog {filename} [clang-args*]")parser.disable_interspersed_args()(opts, args) = parser.parse_args()if len(args) == 0:print 'invalid number arguments'index = Index.create()tu = index.parse(None, args)if not tu:print "unable to load input"output_cursor_and_children(tu.cursor)if __name__ == '__main__':main()

#include "test.h"int main(){Foo f;return 0;
}

#ifndef TEST_H
#define TEST_Hclass Foo 
{int data_;
public:Foo(){}void bar(int data){data_ = data;}
};extern "C" __declspec( dllexport )void test1(){}#endif

How to run:

python linkage_dump.py test.cpp

[Conclusion]

There are so many other ways to make use of ASTs and I wish I had more time to include some of them.  Suffice it to say I’ll probably end up posting about ASTs a few more times.  At least until I work through enough examples to meet my immediate needs.

这篇关于Clang AST parsing for automated code generation的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！