Working Draft: CMake build for Emscripten

CMakeLists.txt

Index: CMakeLists.txt
===================================================================
new file mode 100644
--- /dev/null
+++ b/CMakeLists.txt
@@ -0,0 +1,135 @@
+# Set the minimum we require to the minimum that Emscripten requires.
+# See the Emscripten toolchain file for details. 
+cmake_minimum_required(VERSION 3.4.3)
+
+project (adblockpluscore)
+
+#
+# Sanity check that the tool chain is set up correctly.
+#
+if (NOT "${CMAKE_SYSTEM_NAME}" STREQUAL "Emscripten")
+  message(FATAL_ERROR
+          "The Emscripten toolchain does not appear is not installed in the" 
+          "CMake cache. Please follow the instructions in README.CMAKE.")
+endif()
+
+#
+# Node.js is required to generate the bindings file.
+#
+if (${NODE_JS_EXECUTABLE} MATCHES "NOTFOUND")

[Wladimir Palant, 2017/10/04 12:05:50]

Could it be that Emscripten configures an external Node instance rather than
using its own? That would be suboptimal, and at the very least we'll need to
mention it as an additional dependency.

[Eric, 2017/10/05 13:06:13]

If you do it right, then the Emscripten toolchain file will find its own version
of node.js. If you do it wrong though, it might not find anything.

As you can probably surmise, I did it wrong at first. You have to be inside a
shell with the Emscripten environment when you initially configure CMake. In
that case the toolchain file will locate whatever version of node.js is in the
PATH, and it should be the one from the Emscripten toolchain. The one it finds
goes into the CMake cache and stays there until the cache is cleared.

On the other hand, if you're not in an Emscripten environment, the toolchain
file isn't smart enough to check to see if its being invoked from within its own
distribution tree (instead of, say, being copied elsewhere). In that case,
assuming no other copy of node.js, it finds nothing. You can't generate
bindings, the build will fail, and so the test above is just a sanity test to
make it a bit easier to diagnose not following the directions right.

There is a note about this in the README, but rereading it, it could probably
stand to be expanded and be more prominent.

[sergei, 2017/10/17 10:58:42]

Which node is found actually depends on the order of paths in the PATH variable,
however emscripten configuration has already NODE_JS variable, and it's not
clear why this variable is not used if the environment is required anyway. In
addition, with node.js particularly it would be better to use the exact version
either already configured in emscripten or just mentioned in the documentation,
I'm pretty sure that we could call a python script reading the value and
printing it, though there can be some nuances when it actually happens.

[Eric, 2017/10/17 19:07:22]

Yes, exactly. What you are observing is what I've been dealing with in detail,
which is that the `.cmake` toolchain file that Emscripten provides is, to be
kind about it, immature. It's finicky and has to be called just so. I've
expanded on this issue in the README.
I agree. If we decide to go with CMake, I think it would be advisable to write a
wrapper around their toolchain file that did a better job with everything. It
should be possible to set everything up correctly and reliably just knowing the
root directory of the SDK, but for god-knows-what-reason the Emscripten
developers didn't do it that way.

+  message(FATAL_ERROR
+          "The Node interpreter was not found while setting up the Emscripten"
+          "toolchain.")
+endif()
+
+#
+# Source file lists
+#   We don't use globbing. CMakes expands a globbing expression at the time it
+#   runs, not at the time the build runs. Thus if we were to use globbing, we
+#   would generate build files with fixed lists of sources found at the time
+#   CMake runs. Afterwards, such a build would not discover new files created
+#   in the interim.
+#  
+set(COMMON_SOURCES
+    compiled/bindings/runtime_utils.cpp
+    compiled/filter/ActiveFilter.cpp
+    compiled/filter/BlockingFilter.cpp
+    compiled/filter/CommentFilter.cpp
+    compiled/filter/ElemHideBase.cpp
+    compiled/filter/ElemHideEmulationFilter.cpp
+    compiled/filter/ElemHideException.cpp
+    compiled/filter/ElemHideFilter.cpp
+    compiled/filter/Filter.cpp
+    compiled/filter/InvalidFilter.cpp
+    compiled/filter/RegExpFilter.cpp
+    compiled/filter/WhitelistFilter.cpp
+    compiled/storage/FilterStorage.cpp
+    compiled/subscription/DownloadableSubscription.cpp
+    compiled/subscription/Subscription.cpp
+    compiled/subscription/UserDefinedSubscription.cpp
+    )
+set(CORE_SPECIFIC_SOURCES    
+    compiled/traceInit.cpp
+    )
+set(BINDINGS_SPECIFIC_SOURCES
+    compiled/bindings/generator.cpp
+    compiled/bindings/main.cpp
+    )
+    
+set(BINDINGS_GENERATOR_SOURCES ${COMMON_SOURCES})
+list(APPEND BINDINGS_GENERATOR_SOURCES ${BINDINGS_SPECIFIC_SOURCES})
+list(SORT BINDINGS_GENERATOR_SOURCES)
+
+set(CORE_SOURCES ${COMMON_SOURCES})
+list(APPEND CORE_SOURCES ${CORE_SPECIFIC_SOURCES})
+list(SORT CORE_SOURCES)
+
+#
+# Options for all targets 
+#
+add_compile_options(-std=c++1z)
+
+#
+# Target: bindings-generator
+#
+add_executable(bindings-generator ${BINDINGS_GENERATOR_SOURCES})
+# 
+# CMake overloads `target_link_libraries` for both libraries (as stated) and 
+# for linker flags (very much not as a stated). In order for CMake to 
+# recognize an option with parameter, that is, with an embedded space, it 
+# requires that it be quoted. This oddity is not well-documented in CMake.
+#                      
+target_link_libraries(bindings-generator
+                      "--js-library compiled/library.js"
+                      "--js-library compiled/bindings/library.js"
+                      )
+
+#
+# Target: bindings
+#
+add_custom_target(bindings
+                  COMMAND ${NODE_JS_EXECUTABLE} bindings-generator >bindings.js

[sergei, 2017/10/17 10:58:42]

Is there a way to store both an intermediate and the final results in a special
build/${configuration} subdirectory and how should it be used from
libadblockplus?

[Eric, 2017/10/17 19:07:22]

CMake has a way of exporting projects and importing them into a different
project, along with all the path names. So we don't need to store files in any
particular place; CMake handles that. What we will need to do is to call
export() in this CMakeLists.txt to generate a `.cmake` file that can be imported
by another project.

So you still have to know where the export `.cmake` is, but that's all. And it
can just go in the build root (project root for in-source build; somewhere else
for out-of-source build).

See
https://cmake.org/Wiki/CMake/Tutorials/Exporting_and_Importing_Targets
in the section "Exporting from a Build Tree"

+                  BYPRODUCTS bindings)
+add_dependencies(bindings bindings-generator)
+
+#
+# Target: Core
+#
+add_executable(compiled ${CORE_SOURCES})
+add_dependencies(compiled bindings)
+
+target_compile_options(compiled PRIVATE
+                       -O3 -m32 -Wall -Werror -fno-rtti -fno-exceptions)
+                       
+# Debugging
+target_compile_options(compiled PRIVATE -g3)
+                       
+target_link_libraries(compiled PRIVATE 
+                      "--post-js bindings.js"
+                      -O3
+                      --emit-symbol-map
+                      "--memory-init-file 0"
+                      "-s ASM_JS=2"
+                      "-s TOTAL_MEMORY=16*1024*1024"
+                      "-s TOTAL_STACK=1*1024*1024"
+                      "-s ALLOW_MEMORY_GROWTH=1"
+                      "-s NO_EXIT_RUNTIME=1"
+                      "-s NO_DYNAMIC_EXECUTION=1"
+                      "-s NO_FILESYSTEM=1"
+                      "-s INVOKE_RUN=0"
+                      "-s TEXTDECODER=0"
+                      "-s EXPORTED_RUNTIME_METHODS=\"['cwrap','ccall','stringToAscii']\""
+                      "--js-library compiled/library.js"
+                      )
+#
+# The interaction between command line parsing of "-s SHELL_FILE=" and Windows
+# path names with backslashes is, frankly, insane. It's left as an exercise,
+# that is, waste of time, for the reader to divine all the levels of quoting
+# required to make it work.
+#
+file(GLOB SHELL_FILE compiled/shell.js)
+file(TO_NATIVE_PATH ${SHELL_FILE} SHELL_FILE)
+target_link_libraries(compiled PRIVATE "-s SHELL_FILE=\"\\\"${SHELL_FILE}\\\"\"")

[Wladimir Palant, 2017/10/04 12:05:50]

What about using single quotation marks? SHELL_FILE=\"'${SHELL_FILE}'\"

[Eric, 2017/10/05 13:06:13]

I'll try that. CMake has some capability to do platform-specific calculations,
and I've been thinking that it might be use an ordinary-looking expression as
the default and special-case Windows path names.

From what I can tell, the problem is that Emscripten treats this parameter as
simple text, not as a path name, and also passes it on an internal command line
to invoke other tools without requoting it. This definitely wreaks havoc with
backslashes, but it's probably not the only character that gets maltreated.

It seems that this is related to why Emscripten doesn't run right under CMake
with cygwin.

[Wladimir Palant, 2017/10/09 08:49:12]

It seems that you don't have any escaping issues with single quotation marks
above, in the EXPORTED_RUNTIME_METHODS parameter.Either way, single quotation
marks would be the correct thing to do here anyway. It's not obvious, but this
parameter is actually passing Python code to Emscripten. Python's code style
rule mandate single quotation marks.

[Eric, 2017/10/17 19:07:22]

Done. It works fine.

+
+# Debugging
+target_link_libraries(compiled PRIVATE -g3)