[GCC-XML] gccxml and the latest GCC

[Trevor]

Hello,

I am trying to use gccxml on a large code base to produce XML descriptions of
the individual header files. However, I am running into a problem compiling
code written in the latest C++11. My specific issue is this:

enum A : int
{ ... }

error: expected primary-expression before 'int'
error: expected ';' before 'int'
etc.

So clearly my version of gccxml is using an older version of the gcc C++
compiler. Invoking gcc --version gives me gcc (GCC) 4.7.2, which I thought was
up to date enough to handle the latest C++11 features.

Regardless, I am somewhat confused as to how to tell the gccxml process which
compiler to use. I have tried the flag --gccxml-compiler with cl, which gives
me the exact same problem, even though I can compile the project just fine
within visual studio itself. When I run --gccxml-compiler gcc it tells me it
cannot find gcc (the same thing happens when I enter g++).

For reference, I forked the gccxml depot, used the CMake GUI targeting Visual
Studio 2010, and then built gccxml from the visual studio project created by
CMake. It works just fine on everything but the latest C++11 features.

Can anyone give me some advice on how to resolve this issue?
Thanks very much,
Trevor

[Brad King]

Post by Trevor
I am trying to use gccxml on a large code base to produce XML descriptions of
the individual header files. However, I am running into a problem compiling
code written in the latest C++11.

gccxml is currently based on GCC 4.2's parser and there is no C++11 support.

The --gccxml-compiler option is only for simulating the preprocessor of
another compiler.

There is discussion about updating the internal parser here:

http://thread.gmane.org/gmane.comp.compilers.gccxml/658

but AFAIK no progress has been made since then.

I suggest looking at Clang and its libraries:

http://clang.llvm.org/

as an alternative.

-Brad

[Alex Leach]

Hi,

On Tue, 18 Jun 2013 13:05:39 +0100, Brad King <brad.king at kitware.com>

Post by Brad King
http://thread.gmane.org/gmane.comp.compilers.gccxml/658
but AFAIK no progress has been made since then.

Yea, I've been meaning to get back to you about that... Sorry it's been so
long; you've been very fast to respond yourself.

GCCXML backend upgrade status (to GCC 4.7.2)
---------------------------

So, I was testing on Linux and although I got everything to compile fine,
I couldn't get past certain runtime errors. I recently applied the
upgrades you made to the support headers, but I still couldn't get past
runtime errors mentioning at least `size_type`.

I thought that it was a problem with the Support headers, but I felt
completely out of my depth when reading them; I never read K&R's last
section on compilers and I'm in the middle of writing my PhD thesis (in
Biology!), so I'm afraid I didn't manage to complete the backend upgrade;
I had to abort. I'm sure we could have figured it out together, but I just
ran out of time...

However, I still felt the need to get gccxml working (well, pyplusplus
really) on my system, so I recently spent some time getting GCCXML to work
as a GCC plugin. I started this as a fork a while ago, so I just picked up
where I left off really.

Keeping up with GCC versions
------------------

My system compiler was updated to GCC4.8 some time ago now, which is quite
a major change from GCC 4.7, in that 4.8 now includes C++ code. The most
notable difference, in terms of GCCXML's xml.c, is that the `VEC(tree,
gc)*` macro is now a C++ vector: `vec<tree, va_gc>*`.

Upgrading GCCXML's internal compiler, from GCC 4.7 to 4.8 would again be
quite a major upgrade. As I never got the GCC 4.7 backend to work, I
figured it might be easier to keep up with GCC, if I turned `xml.c` into a
GCC plugin.

So, that's what I did...


Plugin vs standalone gccxml
-----

As a plugin, its nowhere near as platform- and compiler-agnostic as the
main gccxml. It can't, for example, behave as either Intel or Borland
compilers. I also didn't have the confidence to spend my time updating the
compiler driver in C i.e. the front-end `gccxml`, so I (stupidly) re-wrote
it in Python, just so that pygccxml could use the command line arguments
it expects to work (e.g. -fxml).

Getting it to work on Windows seems impossible at this time, without
additional changes to GCC's build system, at least. Although it might be
possible on Cygwin, MinGW does not support GCC plugins. Apparently,
patches have been submitted to enable GCC plugins on Windows, but Andrej,
who should be on this list, has said that even with these patches, he
couldn't get past `./configure --enable-plugin`.

Upsides
-----

Wouldn't be right not to mention what's good about it! For one, it
compiles a lot faster, as there's only two source files: xml.c and
gccxml_plugin.c. It's a lot smaller too; when stripped, the plugin
occupies 92Kb, whereas a debug build (-g3) compiled against a debug build
of GCC occupies 1.3M. (cf. 232kB and 8.3M for stripped, system builds of
gccxml and gccxml_cc1plus, respectively, from master).

---

I can't vouch that the results are identical to those produced by
standalone gccxml, and I'm sure the plugin will be slower, but if you have
GCC 4.8 on Linux, then it should be useful.

So, the GCCXML plugin (appears to) work for my purposes - I generated over
2,000 Boost Python C++ source files with it and pyplusplus yesterday,
which I'm afraid to say, I've failed to do with standalone gccxml and any
GCC > 4.7.2. At the moment, I have the plugin working with GCC 4.8.0 and
4.8.1 on x86_64 Linux, version 3.9.6.

It would require more work to compile against GCC 4.6 and 4.7, as I didn't
wrap these preprocessor macros around all the modifications I made. i.e.

# if GCC_VERSION >= 4008
vec<tree, va_gc>* ...
# else
VEC(tree, gc)* ...
# endif

I'd hope GCC 4.6-7 would work with some more of these pre-processor
macros, but I thought it might be a better idea to instead make separate
tags for each minor GCC version > 4.6. Either way, I'm unlikely to do
either in at least the next few months, as my thesis deadline is
September(!).

---

All the work I've done on gccxml and the plugin has been committed on
github. I'll make sure that's the case after sending this email, anyway..

GCCXML with the GCC-4.7.2 backend can be found here:
https://github.com/alexleach/gccxml_plugin/tree/master
(in case you skipped above text, or I was unclear, this needs more work)

And the gccxml_plugin can be found here:
https://github.com/alexleach/gccxml_plugin
(this works for me, with GCC 4.8.0-1 on Linux)


Brad, I'm sorry that the commit histories are a bit of a mess. You seem
pretty damn good at keeping a clean git history; I've still got a lot to
learn!

I hope that clarifies things, and the work helps someone else other than
me!

If you've got any questions, please feel free to ask. I'll do my best to
answer in a short space of time...

Kind regards,
Alex

[Alex Leach]

On Tue, 18 Jun 2013 15:40:10 +0100, Alex Leach <beamesleach at gmail.com>

Post by Alex Leach
All the work I've done on gccxml and the plugin has been committed on
github. I'll make sure that's the case after sending this email, anyway..

The plugin repo is up to date now; I actually only got it working with
pygccxml yesterday, so some of the changes in the last commit are less
than 24 hours old. Being that new, I would expect there to be more
problems, but all (82) tests now pass; there's one test for each C++
standard library header in GCC 4.8's libstdc++. The tests are only based
on GCC's return code, though. If it's 0, then fine.

Btw, a couple of things I haven't committed, but may be of use:

* gen_xsd.
A basic shell script to generate XSD and DTD representations of the C++
standard library, one of each for C++98 and C++11 headers. It relies on
XML-Schema-learner[1] to generate the XSD and DTD, though, so is very
basic in itself.

* py++
A python script to generate Boost Python C++ code conveniently from the
command line, using pyplusplus and pygccxml[2]. I've designed this so it
can recurse through directories, whilst saving Boost Python sources in a
separate directory tree. pygccxml needs a couple of patches for this to
work though; I used some dirty hacks, but I'll try and get around to
contributing something decent back to the pygccxml maintainer..

If either of the above sound useful, please let me know and I can send
over a copy, or commit them to my github fork..

Kind regards,
Alex



[1]: https://github.com/kore/XML-Schema-learner
[2]: http://sourceforge.net/projects/pygccxml/

[Brad King]

Hi Alex,

Thanks for reporting back on your status.

Post by Alex Leach
On Tue, 18 Jun 2013 15:40:10 +0100, Alex Leach <beamesleach at gmail.com>

Post by Alex Leach
All the work I've done on gccxml and the plugin has been committed on
github. I'll make sure that's the case after sending this email, anyway..

The plugin repo is up to date now; I actually only got it working with
pygccxml yesterday, so some of the changes in the last commit are less
than 24 hours old. Being that new, I would expect there to be more
problems, but all (82) tests now pass; there's one test for each C++
standard library header in GCC 4.8's libstdc++. The tests are only based
on GCC's return code, though. If it's 0, then fine.

This sounds like a good approach and could possibly be useful upstream
as a build option to switch between plugin and standalone modes. I
look forward to taking a look at the work in your repo when I get a
chance (not sure when).

Post by Alex Leach
* gen_xsd.
A basic shell script to generate XSD and DTD representations of the C++
standard library, one of each for C++98 and C++11 headers. It relies on
XML-Schema-learner[1] to generate the XSD and DTD, though, so is very
basic in itself.
* py++
A python script to generate Boost Python C++ code conveniently from the
command line, using pyplusplus and pygccxml[2]. I've designed this so it
can recurse through directories, whilst saving Boost Python sources in a
separate directory tree. pygccxml needs a couple of patches for this to
work though; I used some dirty hacks, but I'll try and get around to
contributing something decent back to the pygccxml maintainer..

These look interesting but are beyond the scope of what I'd like to
maintain in the upstream repo.

Thanks,
-Brad

[Alex Leach]

On Tue, 18 Jun 2013 19:53:23 +0100, Andrej Mitrovic

+ patch the GCC configure script so it builds with plugin support.
What I failed to mention is in the forum thread someone posted
patches, but they're based on GCC 4.6.x (and they work), but
gccxml-plugin is based on 4.8.0 so I need to have a 4.8.0 MinGW build
with plugin support.

That does sound like a lot of work, so I wish you the best of luck with
it! I've got gccxml_plugin to compile and pass all tests against gcc 4.6
and 4.7 now, so hopefully it would work just as well with a successful
windows plugin build.

GCC takes a lot of time to compile on linux too! Its one of the reasons I
thought the plugin was a good idea.. Btw, do you know about make's `-j`
flag? I almost always use it to parallelise the build process one and a
half as many times as I have available CPU cores... e.g. I'd run `make
-j6` on a quad-core cpu. When a build fails, then I'll fall back to `make
VERBOSE=1`, as error messages often get lost when building in parallel.

HTH,
Alex

--