Tclobj - Using C++ Objects with Tcl =================================== This text addresses the configuration and compilation of the `tclobj' package and various system-specific aspects. For an introduction to and tutorial of its interfaces, see the Postscript document in the doc/ subdirectory. Requirements ------------ o A fully installed and operative installation of Tcl, Version 7.5 or above. o An ANSI-compatible C compiler and a ``matching'' C++ compiler. Both compilers should be fom the same vendor, using gcc but a custom C++ compiler will probably fail. o gcc and g++ are highly recommended. o The package makes much more fun on a system that supports dynamic loading. See below. Supported Systems ----------------- Unfortunately, dynamic loading is quite system-dependent. Usually, it is potentially possible, but there are a few catches. The configuration script knows its way around a few rough edges, but it might very well break on systems it has not been tested on. Static linking is possible on all systems. See below for an extensive discussion. Dynamic loading has so far been tested to work on: o Linux/Elf, i386, gcc/g++ o Digital Unix, DECalpha, gcc/g++ o SunOS 5.x, Sparc, gcc/g++ o HP-UX 10.x, HPpa, gcc/g++ or c89/CC o Ultrix 4.x, MIPS, gcc/g++ o AIX 4.x, RS6000, gcc/g++ or xlc/xlC I hope you can make this list grow ... Configuration ------------- The package comes with an autoconf script that tries to determine the system configuration. The script needs to know some details, in particular of your Tcl installation. There are various parameters to the configuration script if it doesn't manage to find the various files by itself. --with-tclsh=dir Gives the location of an executable tclsh. Can either point to a directory to find the program in, or the executable itself. --with-tcl-include=dir The directory where to find --with-tcl-lib=dir The directory where to find the static or shared Tcl library. --with-tcl-config=file The tclConfig.sh file with Tcl's configuration data (created during Tcl's ./configure run and usually installed in the lib directory). --with-tcl-appinit=file The tclAppInit.c sourcecode. Needed in some cases to build a custom tclsh. --with-tcl=dir Find all of the above below this directory. You can just point this parameter to the Tcl distri- bution directory, but the script also looks for the `bin', `include' and `lib' directory below this dir (for example: --with-tcl=/usr/local). --disable-hacks The script knows a few tricks to make dynamic loading work on various systems. This option stops the script from trying them. See below. --disable-dynamic If the configuration script completely freaks out on you while checking for dynamic linking support, you can use this option to enforce static linking. You can also set the following environment variables to override the script's selection. CC Your C compiler. CXX Your C++ compiler. CFLAGS Additional flags to add to the compiler. LDSO How to link shared libraries. LDLIBS Linker options when linking shared libraries. CPICFLAGS Flags to the compiler when compiling files into shared libraries. The script first tries to configure tclobj for build into a dynamically loadable library, and tests whether such a configuration would work. Yet this is a highly system-dependent task. The configuration script has a few tricks to play, especially if you are running gcc and g++, but the author only has access to a limited number of systems, and no doubt the script will fail to determine the correct parameters on many others. As last resort, there is always the possibility to compile the package as static library; this would force you to later statically link the package and all of your classes into a custom tclsh. And this would be a pity, because dynamic loading of new classes is a very desirable feature. But what's the catch? --------------------- On many systems, code must be compiled for position-independence in order to be dynamically loadable. Unlike an executable, which always loads to the same fixed address in memory, the position a dynamic module is loaded to depends on the other memory already used by the same process. This is not yet a problem, there is always a compiler option to do the job (-fPIC for gcc). The problems start when we compile C++ code into dynamically loadable modules. C++ code needs a few external hooks, which are usually provided by some external library. For example, when using g++, the `new' operator invokes the external procedure __builtin_new, which is linked to an exe- cutable from the `libgcc.a' library from your gcc distribution. (If you do not use g++ but some custom compiler, chances are that this step is just the same: the names of the hooks will be different, and they are provided by some system-specific libraries.) This business is usually hidden from your view, but it becomes relevant now, because the libraries which provide the hooks are usually not com- piled for position-independence! So if you have a C++ module, you can either link it into a shared library together with the hooks, meaning that the module won't be loadable due to non-PIC code, or you can not link it with the hooks, in which case the module won't be loadable due to unresolved references. Duh. Fine alternative. However, there are a few workarounds, which the configuration script checks for. Yet the problem it faces is a substantial one, and the script's knowledge is limited. If it fails to find the optimal solution for your system, you may either have to live with static linking, or to hack the Makefile yourself. The author is no expert on shared libraries. What I know, I have learned from experience. I only have experience on a limited number of systems, and do not know about the many problems that this package has to face on others. If you think you can improve the behaviour on some system, let me know. Strategies ---------- The script tries to categorize your system. At the end of its run, it emits a ``warning'' stating the strategy it tries to use. It knows of the following strategies: 1.) No hooks necessary On some systems, the C++ hooks are available by default and need not be linked in from an external library. This saves us of the bother of finding out how to provide the functions to a shared library ourselves, and makes our job as straight- forward as can be. Examples: Linux/ELF 2.) Using gcc/g++ When working with g++, we can cheat by linking in the two object files _eprintf.o and _builtin.o, which provide the necessary functions, mirroring the behaviour of the originals. These two files we can compile for position-independence, thus having all we need. All other subsequently loaded modules can also profit from these exported hooks and don't need to link them in again (making the build of classes as easy as in the first case). Examples: Most systems using gcc/g++ 3.) Using gcc/g++, if _builtin.o does not seem to help. Another solution is to statically link the necessary hooks with the ``parent application'', the tclsh. This strategy extracts the complete contents of libgcc.a, and forces a custom tclsh (branded ``tclobjsh'') to link with all of them. The hooks are then available to all loaded modules. A drawback is that you will have to perform the same trick manually with your wish, should you want to work with X-based applications. A different solution would be to recompile libgcc.a as a shared library. You can hack gcc's Makefile to make it build only this library, compiling the files for position-independence. I have tried this option myself, and it works, but you need to know the right places where to tweak the Makefile ... (this option is not supported) Examples: none known at the moment 99.) Special HP-UX hack On HP-UX, we can work around the problem using the native compilers `c89' and `CC', because their necessary C++ hooks are available as shared library. The trick is that some additional data is required (the cxxhead.o module from /lib/libcxx.a), which we link with the tclobj code. However, the CC compiler, at least in our installation, sucks and can't even compile most example programs. If you prefer living with gcc, use ``configure --disable-hacks'' 0.) Other Systems On other systems, you still have the option of dropping the support for dynamic loading. It hurts, but the main features of the package (using C++ objects from Tcl, remember) still works. Tclobj and all classes will be built into plain old-style libraries (`libtclobj.a'). You will then have to set up your own main function, following the template in tclAppInit.c (from the Tcl distribution), calling the necessary Initialization procedures of all required classes. Then, compile your main file, and link it statically with libtclobj.a and your classes. Examples: none known at the moment The last is the option the configuration script escapes to if it can't make sense of your system with its limited knowledge. But by reading the information about the other strategies, and by some experimenting, you may still get dynamic loading to work. For example, if you do not run gcc/g++, the library that defines the C++ hooks is named differently, but the general trick may still work, so you could just find out what the library is, and then follow the above instructions. With some `hacking', you can still make it work! Global Constructors ------------------- Another troublesome issue are constructors for global objects in shared libraries. They must be called before any of the functions in the module are entered. However, many systems don't do this properly. The config script checks whether global constructors work. Tclobj does not use global objects, and thus it continues even if the constructors are not called, but you must take care of your code. On affected systems, you cannot place code that provides global objects into loadable modules, but must link it statically. Global constructors are known to work on Linux, SunOS 5.x and Digital Unix. They could be made to work on HP-UX 10.x with manual effort, but generally, the answer is no. Global constructors are known not to work on Ultrix and AIX. If you want to use global objects in shared libraries, I really wouldn't blame you if you decided to screw the old systems and decided that people better get a proper OS before trying to run your system. However, there's the workaround of using dynamic objects instead of static ones. Change a global object `FooBar Global' to a pointer, `FooBar *Global', and then during initialization of the package, allocate the object using `new'. Sure, this involves editing all references, but it's portable. You could also hide the redefinition with a macro. Installation ------------ No installation is necessary. There is no ``make install'' yet. If you insist on a global installation, copy the files `tclobjconf.sh', `libtclobj.*' and `tclobj.h' to directories of your choice. System-specific Notes --------------------- Linux/Elf: No problems here, works beautifully. Digital Unix (OSF): Works very well with gcc/g++. Note that you must link modules with `gcc -shared' instead of `ld -shared', or global constructors won't work. SunOS 5.x: Works very well with gcc/g++. Note that you must link modules with `gcc -shared' instead of `ld -shared', or global constructors won't work. The native compilers automatically link with `-lsocket -lnls'. If Tcl was compiled with cc, and you want to compile Tclobj with gcc, these libraries must be manually added to the linker's command line. HP-UX 10.x: Dynamic loading works, but I could not get global constructors to work. There is a hack that makes dynamic loading work using the native c89/CC compilers, which need an additional object linked in from a system library. You *could* get global constructors to work by manually specifying the constructors' names on the linker's command line (using the +I option), but it's a lot of work. Ultrix: Quite surprisingly, dynamic linking is not a problem (using gcc/g++), but global constructors don't work. A hack is to use `-G 0' instead of `-fPIC' even when using gcc. AIX 4.x: In order to make dynamic loading work, you must slightly patch the script that Tcl provides for linking a loadable module, which is among the distribution files as `unix/ldAix'. Edit this file, look for the definition of nmopts, and add the `-C' flag: nmopts="-g -C" ^^^ This prevents nm from demangling C++ symbol names. With this fix, dynamic linking works both with gcc/g++ and with xlc/xlC. In the latter case, the configuration script adds the `-lC' library to the linker command. General Notes ------------- o Mixing code from different C++ compilers will usually not work, because they may implement some internal behaviour differently, for example v-tables. o If you keep receiving "unresolved external" messages when trying to load a class in Tcl, but do not get a list of _what_ symbols are missing, you may want to modify Tcl's module loading code. For example on HP-UX, you can add BIND_VERBOSE to the shl_load() call in unix/tclLoadShl.c o There seems to be a bug in libg++ 2.7.0. If you get an unresolved external of `strchr', you should upgrade libg++ to at least 2.7.1, or you can work around the problem by changing #include at the top of tclobj.cc to `#include "/usr/include/string.h"'. o You will have some fun if you try to use external non-shared libraries. You will either have to recompile them with the proper options for position-independence, or you can perform the same trick as in strategy #3, statically linking the library's contents with a custom tclsh (growing it beyond measure). o ELF systems seem to be able to load shared libraries that have not been compiled for position-independence. However, it seems that this may cause problems if the shared library is mapped more than once, hence this option is discouraged and not supported by default. Mailing List ------------ I have set up a mailing list for tclobj, intended to provide a means of information exchange, or for announcements, should you want to make your Tcl-supporting classes publicly available. To subscribe, send mail to `tclobj-request@belle.fpp.tm.informatik.uni-frankfurt.de' with a subject of `subscribe'. This mailing list is operated on an offline system that polls mail roughly twice a day, so don't be surprised if response time is slow. Examples -------- After compiling the Tclobj module, you can run configure and make in the `example' subdirectory. It uses the pre-determined configuration values stored in tclobjconf.sh and should work quite effortlessly. Don't hesitate to copy the example configure.in for your own projects. On systems that do not support dynamic loading (strategy `0' above), the files are built into object files, and you will have to link them into an executable tclsh in order to test them. The last example, `tclftp' is not built by default, because you probably don't have the socket++ library installed (which also needs the patch from the end of tclftp.c). You will also run into the problem that libsocket++ is usually not compiled for position-independence. You will have to recompile the library as PIC to make tclftp loadable. License ------- Tclobj is free for non-commercial usage. If you decide to use this package in a commercial or shareware project, please contact me first. Credits ------- Thanks to Jan Nijtmans for hints and advice and his implementations of gcc's internal functions (_eprintf.c, _builtin.c). ------------------------------------------------------------ Frank Pilhofer fp@informatik.uni-frankfurt.de