libvorbis and Milepost

8:26 AM Nik 0 Comments

After integrating Milepost into nspr, I found that it was not possible to measure the performance boost there. So Dan suggested that I try the same thing for libvorbis, the ogg library which is used for html5 video. So I started working on that, and as now I'm used to the mozilla build system it didn't take too long to perform the same steps, with minor variations for libvorbis.
Unfortunately, while using Milepost for libvorbis, I am now facing a compilation error on the file vorbis_res0.c. It works perfectly fine as long as I am not using Milepost, but the moment I enable Milepost, it gives a compilation error.
I have informed the cTuning people about this, and a thread is running here. It might just be that its something I am doing wrong, but frankly I can't think of anything. I hope they can give me a solution to the problem soon.
Lastly, I still cannot try checking the perf boost, because the Web Service is still not running properly. I have informed the cTuning people about this too and they are working on it right now. I hope it gets resolved soon, because until it does, I am stuck with nowhere to go.

nspr and Milepost

10:13 AM Nik 0 Comments

I haven't posted for some time now, mainly because I did not have anything concrete, but now I am regretting that. There are so many tweaks here and there that I have done in the past 10 days or so to be able to integrate Milepost GCC into the Mozilla build-system that I am finding it difficult to remember all of them. Still, I'll give here a brief sum up of my progress since the last time.

Well it started with a chat with Dan (dwitte). And he suggested that we start work with SpiderMonkey and try and get Milepost integrated there, because the build system of SpiderMonkey is the same as Mozilla. He also suggested that we try and tweak the code written for PGO which also uses two passes on the system. So off I dived into PGO, got tangled up in makefiles and configures and whatnots and then finally had to approach Ted (ted) to make some sense out of all of it :). And then I got to know about config.mk which sets the CFLAGS and then all the fog cleared :). So I basically figured how PGO was working, but Ted told me that it had been disabled on Linux. Anyways, it helped me a lot in how to call multiple makes etc.

So I started trying to integrate Milepost into SpiderMonkey when I came across this discussion on the cTuning group which said that C++ is not currently supported and I saw that most of the files I was compiling were cpp, so I had run into a dead-end. So then off I went to Dave(humph) and Ted and they gave me names of some C-only modules and humph suggested nspr, a C-only module which provides a platform independent API for system level functions. So then I had to start again with nspr, although it was quite similar to SpiderMonkey and much simpler to understand :).

After much banging heads against the build-system I managed to understand the flow and I now knew what I had to do. So I created a "milepostbuild" target, similar to a "profiledbuild" target of PGO, which would call submakes, changing the environment variables as required by the ICI plugins of Milepost everytime. It was a huge task in itself to find out how to export the variables to the current shell in the first place but I finally managed. I am currently exporting them initially in configure, and to call configure I am using "source ../configure" in bash instead of a normal "./configure". This sets up my variables in the current shell and now I can make whatever changes I want to these variables while calling make again from the makefile by adding VAR=Val alongside.

This done, I now had to get the ICI plugins to work with the build, properly. For this I needed to add the filename of the source file being compiled by gcc to the files that the plugins create, which basically contain the executed passes (.txt) and the static features (.ft). I couldn't find a way to do this at first and I found myself wandering here and there in the gcc code with no idea at all, but after some searching I found the "function_filename" feature which would be able to return the filename. I used it and it worked fine on my small programs, but it did not work when I used the same thing in the build. Finally, I realised that it was because of the relative path of the filename being returned instead of the filename itself, and then I mended that and voila, everything works now!

So, I am now able to extract the gcc executed passes for each nspr file and its respective static features. I have to integrate the web service in now, to predict the flags using the extracted features, but right now there is some problem with the cTuning web services. Also, there is a slight problem regarding the passes for which I need help from the Milepost people. Lastly, although I am currently trying to make this work on a file-to-file basis, what we really need is something that can work on the whole module in one go, as working on each file, especially using a web service, takes a lot of time and as Dwitte mentioned, it won't scale. Already I see that my build time for nspr has changed from seconds to minutes, when I haven't even used the web service yet. I have posted the same concern to the Milepost authors and hope to receive a positive reply...