Issue 2578 - Refactored and added missing hidden import

runserver.spec

 # PyInstaller spec, run "pyinstaller runserver.spec" from repository root to build
 
-a = Analysis(
-  ['runserver.py'],
+# Hidden imports are supposed to be analyzed recursively. However, due to
+# a bug in PyInstaller imports from inside hidden modules aren't considered.
+# https://github.com/pyinstaller/pyinstaller/issues/1086
+def AnalysisWithHiddenImportsWorkaround(scripts, **kwargs):

[Sebastian Noack, 2015/05/22 21:19:27]

Yeah, as the comment says, the hiddenimports passed to Analysis, are supposed to
be analyzed recursively. This bug has recently been fixed in PyInstaller, but
the last release was in 2013. :(

However, we can already achieve that behavior by generating a script importing
the hidden modules, have PyInstaller analyze it, and removing the script
afterwards.

This is has the advantage, that hidden imports aren't performed in adavance. In
particular with modules (like minidom) that are only included for particular
websites, importing them no matter what seemed wrong to me. And once a new
version of PyInstaller is released we can simply remove the workaround.

+  import os
+
+  filename = os.path.join(WORKPATH, '_hidden_imports.py')
+  with open(filename, 'wb') as file:
+    for module in kwargs.pop('hiddenimports'):
+      print >>file, 'import ' + module
+
+  a = Analysis([filename] + scripts, **kwargs)
+  a.scripts -= [('_hidden_imports', None, None)]

[Wladimir Palant, 2015/05/26 10:46:31]

Shouldn't we remove _hidden_imports.py here?

[Sebastian Noack, 2015/05/26 11:13:57]

The first element is the module name, not the file name.

[Wladimir Palant, 2015/05/26 11:24:43]

And I mean the file - the one you created on disk, and that is currently being
left around.

[Sebastian Noack, 2015/05/26 11:37:39]

Along with some other temporary files in the build directory.  That should be
fine.

+  return a
+
+a = AnalysisWithHiddenImportsWorkaround(
+  ['cms/bin/test_server.py'],
   pathex=['.'],
-  hiddenimports=[],
+  hiddenimports=[
+    'markdown.extensions.attr_list',
+
+    # Used by globals/get_browser_versions.py in web.adblockplus.org
+    'xml.dom.minidom',
+  ],
   excludes=[
     'distutils',
     'doctest',
     'ssl',
     '_ssl',
     'werkzeug',
 
     # Mac-specific
     'Carbon',
     'Finder',
     'StdSuites',
   ],
 )
 
 pyz = PYZ(a.pure)
 
 exe = EXE(
   pyz,
   a.scripts,
   a.binaries,
   a.zipfiles,
   a.datas,
   name='runserver',
   debug=False,
   strip=None,
   upx=False,
   console=True
 )