Use GYP to generate MSVS project files

This is still work in progress but I got pretty far. V8 compiles file without Cygwin as long as you don`t use --generator-output parameter - with that parameter the work directory is cleaner but js2c.py parameters get mangled. Had to change our gyp files in a few places to sync them up with the MSVS projects, also had to add configurations to googletest matching all other projects (compiling gets pretty awkward with Visual Studio otherwise). createsolution.bat starts gyp to create libadblockplus.sln and all the project files - target architecture still needs to be parametrized.

[Wladimir Palant, 2013-04-23 16:31:40]

I should mention that at least the x64 abpshell build isn't really functional
for me, it hangs up or crashes when trying to communicate with V8.

[Eric, 2013-04-23 17:12:17]

We're altering third_party/googletest.gyp here. Is this a provisional change, or
have we decided to alter our third-party dependencies?

http://codereview.adblockplus.org/10234088/diff/1/common.gypi
File common.gypi (right):

http://codereview.adblockplus.org/10234088/diff/1/common.gypi#newcode27
common.gypi:27: 'target_arch=="x64"', {
See http://code.google.com/p/mozc/source/browse/trunk/src/gyp/common.gypi for
another way of dealing with architectures. They have * and *_x64 build
configurations all in the same file, which means they get all this in a single
pass, rather than having to run everything twice.

http://codereview.adblockplus.org/10234088/diff/1/createsolution.bat
File createsolution.bat (right):

http://codereview.adblockplus.org/10234088/diff/1/createsolution.bat#newcode1
createsolution.bat:1: @echo off
We can make this more robust by adding the statement "pushd %~dp0" at the
beginning. "pushd" has an implicit "popd" at the end of the batch file. "%~dp0"
is the path where the batch file resides. Adding this line ensures that
"--depth=." always finds the right directory.

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
File libadblockplus.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
I had this as SHARED_INTERMEDIATE_DIRECTORY because then it makes only a single
copy for all configurations.

I also put that directory right off the build directory so I could find it more
easily. The default directory that msvs makes is deeply buried.

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp
File third_party/googletest.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp#...
third_party/googletest.gyp:1: {
If we're going to go down the path of altering our third-party dependencies to
fit, we certainly can.

It would seem, however, that we could get the same effect with an 'include' of
this file and setting configurations outside.

Don't take this observation as a strong argument against this change.

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp#...
third_party/googletest.gyp:25: ]
Does this work to compile our tests correctly? (as opposed to this library
only). I don't recall if the header that fails is required to compile tests or
not.

Regardless, moving this condition into direct_dependent_settings would ensure
that tests get compiled with the definition.

[Wladimir Palant, 2013-04-24 06:01:34]

http://codereview.adblockplus.org/10234088/diff/1/common.gypi
File common.gypi (right):

http://codereview.adblockplus.org/10234088/diff/1/common.gypi#newcode27
common.gypi:27: 'target_arch=="x64"', {
On 2013/04/23 17:12:17, Eric wrote:
> See http://code.google.com/p/mozc/source/browse/trunk/src/gyp/common.gypi for
> another way of dealing with architectures. They have * and *_x64 build
> configurations all in the same file, which means they get all this in a single
> pass, rather than having to run everything twice.

I was merely making sure that this matches what V8 does - we need all the
generated MSVS projects to use the same configurations. I doubt that we can get
V8 to create both x86 and x64 builds together without changing their build files
massively.

http://codereview.adblockplus.org/10234088/diff/1/createsolution.bat
File createsolution.bat (right):

http://codereview.adblockplus.org/10234088/diff/1/createsolution.bat#newcode1
createsolution.bat:1: @echo off
On 2013/04/23 17:12:17, Eric wrote:
> We can make this more robust by adding the statement "pushd %~dp0" at the
> beginning.

Sure, that batch file isn't very advanced - right now it is only there to make
sure I don't have to remember that command line string.

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
File libadblockplus.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
On 2013/04/23 17:12:17, Eric wrote:
> I had this as SHARED_INTERMEDIATE_DIRECTORY because then it makes only a
single
> copy for all configurations.

I'm not really convinced that this is a good idea - that's what I actually had
initially but this file is specific to this particular target, it shouldn't be
shared.

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp
File third_party/googletest.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp#...
third_party/googletest.gyp:1: {
On 2013/04/23 17:12:17, Eric wrote:
> If we're going to go down the path of altering our third-party dependencies to
> fit, we certainly can.

This file isn't part of googletest, that's why it is outside the googletest
repository. Out of the box googletest uses make or CMake to compile - the gyp
file is ours, as are its shortcomings.

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp#...
third_party/googletest.gyp:25: ]
On 2013/04/23 17:12:17, Eric wrote:
> Does this work to compile our tests correctly?

No, it doesn't - at least not on Windows. I'm still in the process of fixing
that file, the include directories need to be specified for both targets. As I
mentioned, this is work in progress.

[Wladimir Palant, 2013-04-24 10:42:15]

I got tests to build correctly, they pass with the exception of some timing
issues I see on Linux as well. abpshell seems to be functional as well now. I'll
spend a bit of time polishing but I think this is pretty close to done.

[Eric, 2013-04-24 13:09:44]

Nothing new to add on this latest round.

Even if it's only-mostly working, if it doesn't cause problems on other
platforms it's worth moving forward with it and pushing it.

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
File libadblockplus.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
On 2013/04/24 06:01:34, Wladimir Palant wrote:
> 
> I'm not really convinced that this is a good idea - that's what I actually had
> initially but this file is specific to this particular target, it shouldn't be
> shared.

If it's specific to a particular target, it should be in a target-specific
directory. The difference between INTERMEDIATE_DIR and SHARED_INTERMEDIATE_DIR
is in configurations, not in targets. Since there's nothing like
configuration-specific arguments in convert_js.py, nor configuration-specific
conversion sources, it can safely be shared between configurations.

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp
File third_party/googletest.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/third_party/googletest.gyp#...
third_party/googletest.gyp:1: {
On 2013/04/24 06:01:34, Wladimir Palant wrote:
> [..] the gyp file is ours, as are its shortcomings.

Ah. Didn't realize that.

Never mind.

[Wladimir Palant, 2013-04-24 14:04:34]

I created a gyp wrapper to monkey-patch _FixPath() in the MSVS generator - quite
a hack of course but IMHO better than having project files everywhere. Maybe we
can somehow convince Google to fix js2c.py instead. Project files are now
generated in the build/ directory and createsolution.bat takes target
architecture as a command line parameter.

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
File libadblockplus.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
From gyp documentation: "INTERMEDIATE_DIR is only guaranteed to be accessible
within a single target". In fact, on Linux this file is being placed into a
target-specific directory.

[Oleksandr, 2013-04-24 14:15:40]

On 2013/04/24 14:04:34, Wladimir Palant wrote:
> I created a gyp wrapper to monkey-patch _FixPath() in the MSVS generator -
quite
> a hack of course but IMHO better than having project files everywhere. Maybe
we
> can somehow convince Google to fix js2c.py instead. Project files are now
> generated in the build/ directory and createsolution.bat takes target
> architecture as a command line parameter.
> 
> http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
> File libadblockplus.gyp (right):
> 
> http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
> libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
> From gyp documentation: "INTERMEDIATE_DIR is only guaranteed to be accessible
> within a single target". In fact, on Linux this file is being placed into a
> target-specific directory.

LGTM

[Eric, 2013-04-24 14:21:56]

> Maybe we can somehow convince Google to fix js2c.py instead.

I've already done it, mostly, barring testing. The changes are small, directly
analogous to what needed to happen for convert_js.py

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp
File libadblockplus.gyp (right):

http://codereview.adblockplus.org/10234088/diff/1/libadblockplus.gyp#newcode40
libadblockplus.gyp:40: '<(INTERMEDIATE_DIR)/adblockplus.js.cpp'
On 2013/04/24 14:04:34, Wladimir Palant wrote:
> From gyp documentation: "INTERMEDIATE_DIR is only guaranteed to be accessible
> within a single target". In fact, on Linux this file is being placed into a
> target-specific directory.

That's what I get for inferring behavior only from the msvs generator, which
does not, in fact, place such files in anything like a target-specific
directory, but rather into a subdirectory of either the 'Debug' or 'Release'
object directories.

http://codereview.adblockplus.org/10234088/diff/8003/msvs_gyp_wrapper.py
File msvs_gyp_wrapper.py (right):

http://codereview.adblockplus.org/10234088/diff/8003/msvs_gyp_wrapper.py#newc...
msvs_gyp_wrapper.py:16: return path
This may well cause a different kind of failure over time if directories do need
to get changed.

No worries, though. I understand this is just an expedient for the moment.

[Wladimir Palant, 2013-04-24 14:51:05]

http://codereview.adblockplus.org/10234088/diff/8003/msvs_gyp_wrapper.py
File msvs_gyp_wrapper.py (right):

http://codereview.adblockplus.org/10234088/diff/8003/msvs_gyp_wrapper.py#newc...
msvs_gyp_wrapper.py:16: return path
This is pretty safe actually - I doubt that we will ever see directories with
such names.

I fact, I tried a more generic change first: assume that anything without dots
and slashes isn't a path. That works fine for V8 but our codebase unfortunately
uses 'include' and 'googletest' as include paths. Not sure whether we can still
suggest a better solution to the gyp developers (like checking whether the paths
actually exist) but getting js2c.py fixed looks like a more promising approach
right now.