docs/overview.txt
author "Yann E. MORIN" <yann.morin.1998@anciens.enib.fr>
Thu Jun 11 21:47:19 2009 +0000 (2009-06-11)
branch1.4
changeset 1451 25d050084e98
parent 1300 7acdd1de5b50
permissions -rw-r--r--
populate: fix installing dynamic linker 'ld.so'

The dynamic linker, ld.so, needs the execute bit to be set.
Detect tht the library being installed is in fact ld.so and
install it with 0755 instead of 0644.

Fix detecting src == dst.

Use a simpler command to copy src -> dst.

Also change echo to printf, get rid of 'echo -n', which is
highly non-portable.


-------- diffstat follows --------
/trunk/scripts/populate.in | 76 43 33 0 +++++++++++++++++++++++++++++-----------------------
1 file changed, 43 insertions(+), 33 deletions(-)
(transplanted from d7ddcb75e0f703e2ba6d17169167356389224870)
yann@1
     1
File.........: overview.txt
yann@197
     2
Content......: Overview of how crosstool-NG works.
yann@92
     3
Copyrigth....: (C) 2007 Yann E. MORIN <yann.morin.1998@anciens.enib.fr>
yann@192
     4
License......: Creative Commons Attribution Share Alike (CC-by-sa), v2.5
yann@92
     5
yann@628
     6
____________________
yann@628
     7
                   /
yann@628
     8
Table Of Content  /
yann@628
     9
_________________/
yann@628
    10
yann@628
    11
yann@628
    12
Introduction
yann@628
    13
History
yann@628
    14
Installing crosstool-NG
yann@628
    15
  Install method
yann@628
    16
  The hacker's way
yann@1048
    17
  Preparing for packaging
yann@837
    18
  Shell completion
yann@628
    19
  Contributed code
yann@628
    20
Configuring crosstool-NG
yann@628
    21
  Interesting config options
yann@628
    22
  Re-building an existing toolchain
yann@628
    23
Running crosstool-NG
yann@628
    24
  Stopping and restarting a build
yann@628
    25
  Testing all toolchains at once
yann@628
    26
  Overriding the number of // jobs
yann@628
    27
Using the toolchain
yann@628
    28
Toolchain types
yann@628
    29
Internals
yann@628
    30
  Makefile front-end
yann@628
    31
  Kconfig parser
yann@628
    32
  Architecture-specific
yann@628
    33
  Adding a new version of a component
yann@628
    34
  Build scripts
yann@628
    35
yann@1
    36
________________
yann@1
    37
               /
yann@1
    38
Introduction  /
yann@1
    39
_____________/
yann@1
    40
yann@1
    41
crosstool-NG aims at building toolchains. Toolchains are an essential component
yann@1
    42
in a software development project. It will compile, assemble and link the code
rpjday@436
    43
that is being developed. Some pieces of the toolchain will eventually end up
yann@1
    44
in the resulting binary/ies: static libraries are but an example.
yann@1
    45
yann@1
    46
So, a toolchain is a very sensitive piece of software, as any bug in one of the
yann@1
    47
components, or a poorly configured component, can lead to execution problems,
yann@1
    48
ranging from poor performance, to applications ending unexpectedly, to
yann@1
    49
mis-behaving software (which more than often is hard to detect), to hardware
rpjday@436
    50
damage, or even to human risks (which is more than regrettable).
yann@1
    51
yann@1
    52
Toolchains are made of different piece of software, each being quite complex
yann@1
    53
and requiring specially crafted options to build and work seamlessly. This
yann@1
    54
is usually not that easy, even in the not-so-trivial case of native toolchains.
yann@1
    55
The work reaches a higher degree of complexity when it comes to cross-
yann@40
    56
compilation, where it can become quite a nightmare...
yann@1
    57
yann@40
    58
Some cross-toolchains exist on the internet, and can be used for general
yann@1
    59
development, but they have a number of limitations:
yann@1
    60
  - they can be general purpose, in that they are configured for the majority:
yann@1
    61
    no optimisation for your specific target,
yann@1
    62
  - they can be prepared for a specific target and thus are not easy to use,
yann@1
    63
    nor optimised for, or even supporting your target,
rpjday@436
    64
  - they often are using aging components (compiler, C library, etc...) not
yann@1
    65
    supporting special features of your shiny new processor;
yann@1
    66
On the other side, these toolchain offer some advantages:
yann@1
    67
  - they are ready to use and quite easy to install and setup,
yann@1
    68
  - they are proven if used by a wide community.
yann@1
    69
yann@1
    70
But once you want to get all the juice out of your specific hardware, you will
yann@197
    71
want to build your own toolchain. This is where crosstool-NG comes into play.
yann@1
    72
rpjday@436
    73
There are also a number of tools that build toolchains for specific needs,
rpjday@436
    74
which are not really scalable. Examples are:
rpjday@436
    75
  - buildroot (buildroot.uclibc.org) whose main purpose is to build root file
yann@1
    76
    systems, hence the name. But once you have your toolchain with buildroot,
yann@1
    77
    part of it is installed in the root-to-be, so if you want to build a whole
yann@1
    78
    new root, you either have to save the existing one as a template and
yann@1
    79
    restore it later, or restart again from scratch. This is not convenient,
yann@1
    80
  - ptxdist (www.pengutronix.de/software/ptxdist), whose purpose is very
yann@1
    81
    similar to buildroot,
yann@702
    82
  - other projects (openembedded.org for example), which are again used to
yann@1
    83
    build root file systems.
yann@1
    84
rpjday@436
    85
crosstool-NG is really targeted at building toolchains, and only toolchains.
yann@1
    86
It is then up to you to use it the way you want.
yann@1
    87
yann@1
    88
___________
yann@1
    89
          /
yann@1
    90
History  /
yann@1
    91
________/
yann@1
    92
rpjday@436
    93
crosstool was first 'conceived' by Dan Kegel, who offered it to the community
yann@1
    94
as a set of scripts, a repository of patches, and some pre-configured, general
yann@1
    95
purpose setup files to be used to configure crosstool. This is available at
yann@203
    96
http://www.kegel.com/crosstool, and the subversion repository is hosted on
yann@203
    97
google at http://code.google.com/p/crosstool/.
yann@1
    98
yann@1
    99
I once managed to add support for uClibc-based toolchains, but it did not make
yann@437
   100
into mainline, mostly because I didn't have time to port the patch forward to
yann@1
   101
the new versions, due in part to the big effort it was taking.
yann@1
   102
yann@1
   103
So I decided to clean up crosstool in the state it was, re-order the things
yann@437
   104
in place, add appropriate support for what I needed, that is uClibc support
yann@437
   105
and a menu-driven configuration, named the new implementation crosstool-NG,
yann@437
   106
(standing for crosstool Next Generation, as many other comunity projects do,
yann@437
   107
and as a wink at the TV series "Star Trek: The Next Generation" ;-) ) and
yann@437
   108
made it available to the community, in case it was of interest to any one.
yann@1
   109
yann@294
   110
___________________________
yann@294
   111
                          /
yann@294
   112
Installing crosstool-NG  /
yann@294
   113
________________________/
yann@294
   114
yann@294
   115
There are two ways you can use crosstool-NG:
yann@294
   116
 - build and install it, then get rid of the sources like you'd do for most
yann@294
   117
   programs,
yann@294
   118
 - or only build it and run from the source directory.
yann@294
   119
yann@294
   120
The former should be used if you got crosstool-NG from a packaged tarball, see
rpjday@436
   121
"Install method", below, while the latter is most useful for developpers that
yann@294
   122
checked the code out from SVN, and want to submit patches, see "The Hacker's
yann@294
   123
way", below.
yann@294
   124
yann@294
   125
Install method |
yann@294
   126
---------------+
yann@294
   127
yann@294
   128
If you go for the install, then you just follow the classical, but yet easy
yann@294
   129
./configure way:
yann@294
   130
  ./configure --prefix=/some/place
yann@294
   131
  make
yann@294
   132
  make install
yann@294
   133
  export PATH="${PATH}:/some/place/bin"
yann@294
   134
yann@294
   135
You can then get rid of crosstool-NG source. Next create a directory to serve
yann@294
   136
as a working place, cd in there and run:
yann@294
   137
  ct-ng help
yann@294
   138
yann@294
   139
See below for complete usage.
yann@294
   140
yann@294
   141
The Hacker's way |
yann@294
   142
-----------------+
yann@294
   143
yann@294
   144
If you go the hacker's way, then the usage is a bit different, although very
yann@294
   145
simple:
yann@294
   146
  ./configure --local
yann@294
   147
  make
yann@294
   148
yann@294
   149
Now, *do not* remove crosstool-NG sources. They are needed to run crosstool-NG!
yann@294
   150
Stay in the directory holding the sources, and run:
yann@294
   151
  ./ct-ng help
yann@294
   152
yann@294
   153
See below for complete usage.
yann@294
   154
yann@294
   155
Now, provided you checked-out the code, you can send me your interesting changes
yann@294
   156
by running:
yann@294
   157
  svn diff
yann@294
   158
yann@294
   159
and mailing me the result! :-P
yann@294
   160
yann@1048
   161
Preparing for packaging |
yann@1048
   162
------------------------+
yann@1048
   163
yann@1048
   164
If you plan on packaging crosstool-NG, you surely don't want to install it
yann@1048
   165
in your root file system. The install procedure of crosstool-NG honors the
yann@1048
   166
DESTDIR variable:
yann@1048
   167
yann@1048
   168
  ./configure --prefix=/usr
yann@1048
   169
  make
rpjday@1291
   170
  make DESTDIR=/packaging/place install
yann@1048
   171
yann@837
   172
Shell completion |
yann@837
   173
-----------------+
yann@837
   174
yann@837
   175
crosstool-NG comes with a shell script fragment that defines bash-compatible
yann@837
   176
completion. That shell fragment is currently not installed automatically, but
yann@837
   177
this is planned.
yann@837
   178
yann@837
   179
To install the shell script fragment, you have two options:
yann@837
   180
 - install system-wide, most probably by copying ct-ng.comp into
yann@837
   181
   /etc/bash_completion.d/
yann@837
   182
 - install for a single user, by copying ct-ng.comp into ${HOME}/ and
yann@837
   183
   sourcing this file from your ${HOME}/.bashrc
yann@837
   184
yann@456
   185
Contributed code |
yann@456
   186
-----------------+
yann@456
   187
yann@456
   188
Some people contibuted code that couldn't get merged for various reasons. This
yann@456
   189
code is available as patches in the contrib/ sub-directory. These patches are
yann@456
   190
to be applied to the source of crosstool-NG, prior to installing.
yann@456
   191
yann@620
   192
An easy way to use contributed code is to pass the --with-contrib= option to
yann@620
   193
./configure. The possible values depend upon which contributions are packaged
yann@620
   194
with your version, but you can get with it with passing one of those two
yann@620
   195
special values:
yann@620
   196
  --with-contrib=list
yann@620
   197
    will list all available contributions
yann@620
   198
yann@620
   199
  --with-contrib=all
yann@620
   200
    will select all avalaible contributions
yann@620
   201
yann@620
   202
There is no guarantee that a particuliar contribution applies to the current
yann@620
   203
version of crosstool-ng, or that it will work at all. Use contributions at
yann@620
   204
your own risk.
yann@620
   205
yann@168
   206
____________________________
yann@168
   207
                           /
yann@168
   208
Configuring crosstool-NG  /
yann@168
   209
_________________________/
yann@168
   210
yann@620
   211
crosstool-NG is configured with a configurator presenting a menu-stuctured set
yann@620
   212
of options. These options let you specify the way you want your toolchain
yann@620
   213
built, where you want it installed, what architecture and specific processor it
yann@277
   214
will support, the version of the components you want to use, etc... The
yann@277
   215
value for those options are then stored in a configuration file.
yann@277
   216
yann@476
   217
The configurator works the same way you configure your Linux kernel. It is
yann@277
   218
assumed you now how to handle this.
yann@168
   219
yann@168
   220
To enter the menu, type:
yann@192
   221
  ct-ng menuconfig
yann@168
   222
yann@203
   223
Almost every config item has a help entry. Read them carefully.
yann@168
   224
yann@168
   225
String and number options can refer to environment variables. In such a case,
yann@192
   226
you must use the shell syntax: ${VAR}. You shall neither single- nor double-
yann@294
   227
quote the string/number options.
yann@168
   228
yann@192
   229
There are three environment variables that are computed by crosstool-NG, and
yann@168
   230
that you can use:
yann@168
   231
yann@168
   232
CT_TARGET:
yann@335
   233
  It represents the target tuple you are building for. You can use it for
yann@168
   234
  example in the installation/prefix directory, such as:
yann@168
   235
    /opt/x-tools/${CT_TARGET}
yann@168
   236
yann@168
   237
CT_TOP_DIR:
yann@182
   238
  The top directory where crosstool-NG is running. You shouldn't need it in
yann@182
   239
  most cases. There is one case where you may need it: if you have local
yann@182
   240
  patches and you store them in your running directory, you can refer to them
yann@168
   241
  by using CT_TOP_DIR, such as:
yann@168
   242
    ${CT_TOP_DIR}/patches.myproject
yann@168
   243
yann@168
   244
CT_VERSION:
yann@192
   245
  The version of crosstool-NG you are using. Not much use for you, but it's
yann@168
   246
  there if you need it.
yann@168
   247
yann@168
   248
Interesting config options |
yann@476
   249
---------------------------+
yann@168
   250
yann@168
   251
CT_LOCAL_TARBALLS_DIR:
yann@277
   252
  If you already have some tarballs in a direcotry, enter it here. That will
yann@197
   253
  speed up the retrieving phase, where crosstool-NG would otherwise download
yann@168
   254
  those tarballs.
yann@168
   255
yann@168
   256
CT_PREFIX_DIR:
yann@168
   257
  This is where the toolchain will be installed in (and for now, where it
yann@437
   258
  will run from). Common use is to add the target tuple in the directory
yann@277
   259
  path, such as (see above):
yann@277
   260
    /opt/x-tools/${CT_TARGET}
yann@168
   261
yann@168
   262
CT_TARGET_VENDOR:
yann@168
   263
  An identifier for your toolchain, will take place in the vendor part of the
yann@335
   264
  target tuple. It shall *not* contain spaces or dashes. Usually, keep it
yann@168
   265
  to a one-word string, or use underscores to separate words if you need.
yann@168
   266
  Avoid dots, commas, and special characters.
yann@168
   267
yann@168
   268
CT_TARGET_ALIAS:
yann@168
   269
  An alias for the toolchian. It will be used as a prefix to the toolchain
yann@168
   270
  tools. For example, you will have ${CT_TARGET_ALIAS}-gcc
yann@168
   271
yann@246
   272
Also, if you think you don't see enough versions, you can try to enable one of
yann@246
   273
those:
yann@246
   274
yann@246
   275
CT_OBSOLETE:
yann@246
   276
  Show obsolete versions or tools. Most of the time, you don't want to base
yann@246
   277
  your toolchain on too old a version (of gcc, for example). But at times, it
yann@246
   278
  can come handy to use such an old version for regression tests. Those old
yann@1300
   279
  versions are hidden behind CT_OBSOLETE. Those versions (or features) are so
yann@1300
   280
  marked because maintaining support for those in crosstool-NG would be too
yann@1300
   281
  costly, time-wise, and time is dear.
yann@246
   282
yann@246
   283
CT_EXPERIMENTAL:
yann@246
   284
  Show experimental versions or tools. Again, you might not want to base your
yann@246
   285
  toolchain on too recent tools (eg. gcc) for production. But if you need a
yann@246
   286
  feature present only in a recent version, or a new tool, you can find them
yann@1300
   287
  hidden behind CT_EXPERIMENTAL. Those versions (or features) did not (yet)
yann@1300
   288
  receive thorough testing in crosstool-NG, and/or are not mature enough to
yann@1300
   289
  be blindly trusted.
yann@246
   290
yann@276
   291
Re-building an existing toolchain |
yann@276
   292
----------------------------------+
yann@276
   293
yann@276
   294
If you have an existing toolchain, you can re-use the options used to build it
yann@276
   295
to create a new toolchain. That needs a very little bit of effort on your side
yann@894
   296
but is quite easy. The options to build a toolchain are saved with the
yann@894
   297
toolchain, and you can retrieve this configuration by running:
yann@894
   298
  ${CT_TARGET}-config
yann@276
   299
yann@894
   300
This will dump the configuration to stdout, so to rebuild a toolchain with this
yann@894
   301
configuration, the following is all you need to do:
yann@894
   302
  ${CT_TARGET}-config >.config
yann@1098
   303
  ct-ng oldconfig
yann@276
   304
yann@894
   305
Then, you can review and change the configuration by running:
yann@894
   306
  ct-ng menuconfig
yann@276
   307
yann@168
   308
________________________
yann@168
   309
                       /
yann@168
   310
Running crosstool-NG  /
yann@168
   311
_____________________/
yann@1
   312
yann@168
   313
To build the toolchain, simply type:
yann@203
   314
  ct-ng build
yann@135
   315
yann@135
   316
This will use the above configuration to retrieve, extract and patch the
yann@135
   317
components, build, install and eventually test your newly built toolchain.
yann@1
   318
yann@1
   319
You are then free to add the toolchain /bin directory in your PATH to use
yann@1
   320
it at will.
yann@1
   321
yann@135
   322
In any case, you can get some terse help. Just type:
yann@192
   323
  ct-ng help
yann@203
   324
or:
yann@203
   325
  man 1 ct-ng
yann@135
   326
rpjday@436
   327
Stopping and restarting a build |
yann@476
   328
--------------------------------+
yann@135
   329
yann@135
   330
If you want to stop the build after a step you are debugging, you can pass the
yann@135
   331
variable STOP to make:
yann@192
   332
  ct-ng STOP=some_step
yann@135
   333
yann@135
   334
Conversely, if you want to restart a build at a specific step you are
yann@135
   335
debugging, you can pass the RESTART variable to make:
yann@192
   336
  ct-ng RESTART=some_step
yann@135
   337
yann@136
   338
Alternatively, you can call make with the name of a step to just do that step:
yann@192
   339
  ct-ng libc_headers
yann@136
   340
is equivalent to:
yann@620
   341
  ct-ng RESTART=libc_headers STOP=libc_headers
yann@136
   342
yann@304
   343
The shortcuts +step_name and step_name+ allow to respectively stop or restart
yann@136
   344
at that step. Thus:
yann@304
   345
  ct-ng +libc_headers        and:    ct-ng libc_headers+
yann@136
   346
are equivalent to:
yann@192
   347
  ct-ng STOP=libc_headers    and:    ct-ng RESTART=libc_headers
yann@136
   348
yann@181
   349
To obtain the list of acceptable steps, please call:
yann@544
   350
  ct-ng list-steps
yann@181
   351
yann@168
   352
Note that in order to restart a build, you'll have to say 'Y' to the config
yann@168
   353
option CT_DEBUG_CT_SAVE_STEPS, and that the previous build effectively went
yann@168
   354
that far.
yann@92
   355
yann@1025
   356
Building all toolchains at once |
yann@1025
   357
--------------------------------+
yann@92
   358
yann@1025
   359
You can build all samples; simply call:
yann@1025
   360
  ct-ng build-all
yann@40
   361
yann@335
   362
Overriding the number of // jobs |
yann@476
   363
---------------------------------+
yann@335
   364
yann@335
   365
If you want to override the number of jobs to run in // (the -j option to
yann@335
   366
make), you can either re-enter the menuconfig, or simply add it on the command
yann@335
   367
line, as such:
yann@335
   368
  ct-ng build.4
yann@335
   369
yann@335
   370
which tells crosstool-NG to override the number of // jobs to 4.
yann@335
   371
yann@335
   372
You can see the actions that support overriding the number of // jobs in
yann@335
   373
the help menu. Those are the ones with [.#] after them (eg. build[.#] or
yann@1025
   374
build-all[.#], and so on...).
yann@1025
   375
yann@1025
   376
Note on // jobs |
yann@1025
   377
----------------+
yann@1025
   378
yann@1025
   379
The crosstool-NG script 'ct-ng' is a Makefile-script. It does *not* execute
yann@1025
   380
in parallel (there is not much to gain). When speaking of // jobs, we are
yann@1025
   381
refering to the number of // jobs when making the *components*. That is, we
yann@1025
   382
speak of the number of // jobs used to build gcc, glibc, and so on...
yann@1025
   383
yann@335
   384
yann@227
   385
_______________________
yann@227
   386
                      /
yann@227
   387
Using the toolchain  /
yann@227
   388
____________________/
yann@227
   389
yann@227
   390
Using the toolchain is as simple as adding the toolchain's bin directory in
yann@227
   391
your PATH, such as:
yann@227
   392
  export PATH="${PATH}:/your/toolchain/path/bin"
yann@227
   393
yann@335
   394
and then using the target tuple to tell the build systems to use your
yann@227
   395
toolchain:
yann@335
   396
  ./configure --target=your-target-tuple
yann@294
   397
or
yann@335
   398
  make CC=your-target-tuple-gcc
yann@294
   399
or
yann@335
   400
  make CROSS_COMPILE=your-target-tuple-
yann@294
   401
and so on...
yann@227
   402
yann@476
   403
It is strongly advised not to use the toolchain sys-root directory as an
yann@620
   404
install directory for your programs/packages. If you do so, you will not be
yann@476
   405
able to use your toolchain for another project. It is even strongly advised
yann@476
   406
that your toolchain is chmod-ed to read-only once successfully build, so that
yann@620
   407
you don't go polluting your toolchain with your programs/packages' files.
yann@476
   408
yann@476
   409
Thus, when you build a program/package, install it in a separate directory,
yann@476
   410
eg. /your/root. This directory is the /image/ of what would be in the root file
yann@620
   411
system of your target, and will contain all that your programs/packages have
yann@476
   412
installed.
yann@476
   413
yann@227
   414
When your root directory is ready, it is still missing some important bits: the
yann@227
   415
toolchain's libraries. To populate your root directory with those libs, just
yann@227
   416
run:
yann@335
   417
  your-target-tuple-populate -s /your/root -d /your/root-populated
yann@227
   418
yann@227
   419
This will copy /your/root into /your/root-populated, and put the needed and only
yann@227
   420
the needed libraries there. Thus you don't polute /your/root with any cruft that
yann@227
   421
would no longer be needed should you have to remove stuff. /your/root always
yann@227
   422
contains only those things you install in it.
yann@227
   423
yann@227
   424
You can then use /your/root-populated to build up your file system image, a
yann@227
   425
tarball, or to NFS-mount it from your target, or whatever you need.
yann@227
   426
yann@294
   427
populate accepts the following options:
yann@294
   428
yann@294
   429
 -s [src_dir]
yann@294
   430
    Use 'src_dir' as the 'source', un-populated root directory
yann@294
   431
yann@294
   432
 -d [dst_dir]
yann@294
   433
    Put the 'destination', populated root directory in 'dst_dir'
yann@294
   434
yann@294
   435
 -f
yann@294
   436
    Remove 'dst_dir' if it previously existed
yann@294
   437
yann@294
   438
 -v
yann@294
   439
    Be verbose, and tell what's going on (you can see exactly where libs are
yann@294
   440
    coming from).
yann@294
   441
yann@294
   442
 -h
yann@294
   443
    Print the help
yann@294
   444
yann@40
   445
___________________
yann@40
   446
                  /
yann@40
   447
Toolchain types  /
yann@40
   448
________________/
yann@40
   449
yann@40
   450
There are four kinds of toolchains you could encounter.
yann@40
   451
yann@40
   452
First off, you must understand the following: when it comes to compilers there
yann@40
   453
are up to four machines involved:
yann@40
   454
  1) the machine configuring the toolchain components: the config machine
yann@40
   455
  2) the machine building the toolchain components:    the build machine
yann@40
   456
  3) the machine running the toolchain:                the host machine
yann@203
   457
  4) the machine the toolchain is generating code for: the target machine
yann@40
   458
yann@40
   459
We can most of the time assume that the config machine and the build machine
yann@40
   460
are the same. Most of the time, this will be true. The only time it isn't
yann@40
   461
is if you're using distributed compilation (such as distcc). Let's forget
yann@40
   462
this for the sake of simplicity.
yann@40
   463
yann@40
   464
So we're left with three machines:
yann@40
   465
 - build
yann@40
   466
 - host
yann@40
   467
 - target
yann@40
   468
yann@40
   469
Any toolchain will involve those three machines. You can be as pretty sure of
yann@40
   470
this as "2 and 2 are 4". Here is how they come into play:
yann@40
   471
yann@40
   472
1) build == host == target
yann@40
   473
    This is a plain native toolchain, targetting the exact same machine as the
yann@40
   474
    one it is built on, and running again on this exact same machine. You have
yann@40
   475
    to build such a toolchain when you want to use an updated component, such
yann@40
   476
    as a newer gcc for example.
yann@197
   477
    crosstool-NG calls it "native".
yann@40
   478
yann@40
   479
2) build == host != target
yann@40
   480
    This is a classic cross-toolchain, which is expected to be run on the same
yann@40
   481
    machine it is compiled on, and generate code to run on a second machine,
yann@40
   482
    the target.
yann@197
   483
    crosstool-NG calls it "cross".
yann@40
   484
yann@40
   485
3) build != host == target
yann@40
   486
    Such a toolchain is also a native toolchain, as it targets the same machine
yann@40
   487
    as it runs on. But it is build on another machine. You want such a
yann@40
   488
    toolchain when porting to a new architecture, or if the build machine is
yann@40
   489
    much faster than the host machine.
yann@197
   490
    crosstool-NG calls it "cross-native".
yann@40
   491
yann@40
   492
4) build != host != target
yann@92
   493
    This one is called a canadian-toolchain (*), and is tricky. The three
yann@40
   494
    machines in play are different. You might want such a toolchain if you
yann@40
   495
    have a fast build machine, but the users will use it on another machine,
yann@40
   496
    and will produce code to run on a third machine.
yann@197
   497
    crosstool-NG calls it "canadian".
yann@40
   498
yann@197
   499
crosstool-NG can build all these kinds of toolchains (or is aiming at it,
yann@197
   500
anyway!)
yann@40
   501
yann@40
   502
(*) The term Canadian Cross came about because at the time that these issues
yann@40
   503
    were all being hashed out, Canada had three national political parties.
yann@40
   504
    http://en.wikipedia.org/wiki/Cross_compiler
yann@40
   505
yann@1
   506
_____________
yann@1
   507
            /
yann@1
   508
Internals  /
yann@1
   509
__________/
yann@1
   510
yann@92
   511
Internally, crosstool-NG is script-based. To ease usage, the frontend is
yann@92
   512
Makefile-based.
yann@92
   513
yann@92
   514
Makefile front-end |
yann@476
   515
-------------------+
yann@92
   516
yann@203
   517
The entry point to crosstool-NG is the Makefile script "ct-ng". Calling this
yann@203
   518
script with an action will act exactly as if the Makefile was in the current
yann@203
   519
working directory and make was called with the action as rule. Thus:
yann@203
   520
  ct-ng menuconfig
yann@294
   521
yann@203
   522
is equivalent to having the Makefile in CWD, and calling:
yann@203
   523
  make menuconfig
yann@203
   524
yann@203
   525
Having ct-ng as it is avoids copying the Makefile everywhere, and acts as a
yann@203
   526
traditional command.
yann@203
   527
yann@203
   528
ct-ng loads sub- Makefiles from the library directory $(CT_LIB_DIR), as set up
yann@203
   529
at configuration time with ./configure.
yann@203
   530
yann@437
   531
ct-ng also searches for config files, sub-tools, samples, scripts and patches in
yann@203
   532
that library directory.
yann@92
   533
yann@294
   534
Because of a stupid make behavior/bug I was unable to track down, implicit make
yann@294
   535
rules are disabled: installing with --local would triger those rules, and mconf
yann@294
   536
was unbuildable.
yann@294
   537
yann@182
   538
Kconfig parser |
yann@476
   539
---------------+
yann@92
   540
yann@965
   541
The kconfig language is a hacked version, vampirised from the Linux kernel
yann@965
   542
(http://www.kernel.org/), and (heavily) adapted to my needs.
yann@92
   543
yann@1040
   544
The list of the most notable changes (at least the ones I remember) follows:
yann@1040
   545
- the CONFIG_ prefix has been replaced with CT_
yann@1040
   546
- a leading | in prompts is skipped, and subsequent leading spaces are not
yann@1040
   547
  trimmed
yann@1040
   548
- otherwise leading spaces are silently trimmed
yann@1040
   549
yann@203
   550
The kconfig parsers (conf and mconf) are not installed pre-built, but as
yann@203
   551
source files. Thus you can have the directory where crosstool-NG is installed,
yann@203
   552
exported (via NFS or whatever) and have clients with different architectures
yann@203
   553
use the same crosstool-NG installation, and most notably, the same set of
yann@203
   554
patches.
yann@203
   555
yann@381
   556
Architecture-specific |
yann@476
   557
----------------------+
yann@381
   558
yann@628
   559
Note: this chapter is not really well written, and might thus be a little bit
yann@628
   560
complex to understand. To get a better grasp of what an architecture is, the
yann@628
   561
reader is kindly encouraged to look at the "arch/" sub-directory, and to the
yann@628
   562
existing architectures to see how things are laid out.
yann@628
   563
yann@381
   564
An architecture is defined by:
yann@381
   565
yann@381
   566
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@628
   567
   The underscore is allowed; space and special characters are not.
yann@628
   568
     Eg.: arm, x86_64
yann@903
   569
 - a file in "config/arch/", named after the architecture's name, and suffixed
yann@903
   570
   with ".in".
yann@903
   571
     Eg.: config/arch/arm.in
yann@903
   572
 - a file in "scripts/build/arch/", named after the architecture's name, and
yann@903
   573
   suffixed with ".sh".
yann@903
   574
     Eg.: scripts/build/arch/arm.sh
yann@628
   575
yann@903
   576
The architecture's ".in" file API:
yann@628
   577
 > the config option "ARCH_%arch%" (where %arch% is to be replaced with the
yann@628
   578
   actual architecture name).
yann@628
   579
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@628
   580
   *not* depend on any other config option (EXPERIMENTAL is managed as above).
yann@628
   581
     Eg.:
yann@628
   582
       config ARCH_arm
yann@630
   583
   + mandatory:
yann@702
   584
       defines a (terse) help entry for this architecture:
yann@630
   585
       Eg.:
yann@630
   586
         config ARCH_arm
yann@630
   587
           help
yann@630
   588
             The ARM architecture.
yann@628
   589
   + optional:
yann@628
   590
       selects adequate associated config options.
yann@1038
   591
       Note: 64-bit architectures *shall* select ARCH_64
yann@628
   592
       Eg.:
yann@628
   593
         config ARCH_arm
yann@628
   594
           select ARCH_SUPPORTS_BOTH_ENDIAN
yann@628
   595
           select ARCH_DEFAULT_LE
yann@630
   596
           help
yann@630
   597
             The ARM architecture.
yann@1038
   598
       Eg.:
yann@1038
   599
         config ARCH_x86_64
yann@1038
   600
            select ARCH_64
yann@1038
   601
            help
yann@1038
   602
              The x86_64 architecture.
yann@628
   603
yann@628
   604
 > other target-specific options, at your discretion. Note however that to
yann@628
   605
   avoid name-clashing, such options shall be prefixed with "ARCH_%arch%",
yann@628
   606
   where %arch% is again replaced by the actual architecture name.
yann@628
   607
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@628
   608
    I may have left some config options that do not completely conform to
yann@628
   609
    this, as the architecture name was written all upper case. However, the
yann@628
   610
    prefix is unique among architectures, and does not cause harm).
yann@381
   611
yann@903
   612
The architecture's ".sh" file API:
yann@965
   613
 > the function "CT_DoArchTupleValues"
yann@381
   614
   + parameters: none
yann@381
   615
   + environment:
yann@901
   616
     - all variables from the ".config" file,
yann@901
   617
     - the two variables "target_endian_eb" and "target_endian_el" which are
yann@901
   618
       the endianness suffixes
yann@381
   619
   + return value: 0 upon success, !0 upon failure
yann@381
   620
   + provides:
yann@391
   621
     - mandatory
yann@383
   622
     - the environment variable CT_TARGET_ARCH
yann@389
   623
     - contains:
yann@389
   624
       the architecture part of the target tuple.
yann@389
   625
       Eg.: "armeb" for big endian ARM
yann@389
   626
            "i386" for an i386
yann@389
   627
   + provides:
yann@391
   628
     - optional
yann@389
   629
     - the environment variable CT_TARGET_SYS
yann@456
   630
     - contains:
yann@383
   631
       the sytem part of the target tuple.
yann@383
   632
       Eg.: "gnu" for glibc on most architectures
yann@383
   633
            "gnueabi" for glibc on an ARM EABI
yann@383
   634
     - defaults to:
yann@383
   635
       - for glibc-based toolchain: "gnu"
yann@383
   636
       - for uClibc-based toolchain: "uclibc"
yann@383
   637
   + provides:
yann@383
   638
     - optional
yann@391
   639
     - the environment variable CT_KERNEL_ARCH
yann@383
   640
     - contains:
yann@391
   641
       the architecture name as understandable by the Linux kernel build
yann@391
   642
       system.
yann@391
   643
       Eg.: "arm" for an ARM
yann@391
   644
            "powerpc" for a PowerPC
yann@391
   645
            "i386" for an x86
yann@383
   646
     - defaults to:
yann@391
   647
       ${CT_ARCH}
yann@391
   648
   + provides:
yann@391
   649
     - optional
yann@767
   650
     - the environment variables to configure the cross-gcc (defaults)
yann@767
   651
       - CT_ARCH_WITH_ARCH    : the gcc ./configure switch to select architecture level         ( "--with-arch=${CT_ARCH_ARCH}"   )
yann@767
   652
       - CT_ARCH_WITH_ABI     : the gcc ./configure switch to select ABI level                  ( "--with-abi=${CT_ARCH_ABI}"     )
yann@767
   653
       - CT_ARCH_WITH_CPU     : the gcc ./configure switch to select CPU instruction set        ( "--with-cpu=${CT_ARCH_CPU}"     )
yann@767
   654
       - CT_ARCH_WITH_TUNE    : the gcc ./configure switch to select scheduling                 ( "--with-tune=${CT_ARCH_TUNE}"   )
yann@767
   655
       - CT_ARCH_WITH_FPU     : the gcc ./configure switch to select FPU type                   ( "--with-fpu=${CT_ARCH_FPU}"     )
yann@767
   656
       - CT_ARCH_WITH_FLOAT   : the gcc ./configure switch to select floating point arithmetics ( "--with-float=soft" or /empty/  )
yann@391
   657
   + provides:
yann@391
   658
     - optional
yann@767
   659
     - the environment variables to pass to the cross-gcc to build target binaries (defaults)
yann@391
   660
       - CT_ARCH_ARCH_CFLAG   : the gcc switch to select architecture level                     ( "-march=${CT_ARCH_ARCH}"            )
yann@456
   661
       - CT_ARCH_ABI_CFLAG    : the gcc switch to select ABI level                              ( "-mabi=${CT_ARCH_ABI}"              )
yann@391
   662
       - CT_ARCH_CPU_CFLAG    : the gcc switch to select CPU instruction set                    ( "-mcpu=${CT_ARCH_CPU}"              )
yann@391
   663
       - CT_ARCH_TUNE_CFLAG   : the gcc switch to select scheduling                             ( "-mtune=${CT_ARCH_TUNE}"            )
yann@391
   664
       - CT_ARCH_FPU_CFLAG    : the gcc switch to select FPU type                               ( "-mfpu=${CT_ARCH_FPU}"              )
yann@391
   665
       - CT_ARCH_FLOAT_CFLAG  : the gcc switch to choose floating point arithmetics             ( "-msoft-float" or /empty/           )
yann@391
   666
       - CT_ARCH_ENDIAN_CFLAG : the gcc switch to choose big or little endian                   ( "-mbig-endian" or "-mlittle-endian" )
yann@391
   667
     - default to:
yann@391
   668
       see above.
yann@767
   669
   + provides:
yann@767
   670
     - optional
yann@767
   671
     - the environement variables to configure the core and final compiler, specific to this architecture:
yann@767
   672
       - CT_ARCH_CC_CORE_EXTRA_CONFIG   : additional, architecture specific core gcc ./configure flags
yann@767
   673
       - CT_ARCH_CC_EXTRA_CONFIG        : additional, architecture specific final gcc ./configure flags
yann@767
   674
     - default to:
yann@767
   675
       - all empty
yann@767
   676
   + provides:
yann@767
   677
     - optional
yann@767
   678
     - the architecture-specific CFLAGS and LDFLAGS:
yann@767
   679
       - CT_ARCH_TARGET_CLFAGS
yann@767
   680
       - CT_ARCH_TARGET_LDFLAGS
yann@767
   681
     - default to:
yann@767
   682
       - all empty
yann@628
   683
yann@903
   684
You can have a look at "config/arch/arm.in" and "scripts/build/arch/arm.sh" for
yann@903
   685
a quite complete example of what an actual architecture description looks like.
yann@901
   686
yann@890
   687
Kernel specific |
yann@890
   688
----------------+
yann@890
   689
yann@890
   690
A kernel is defined by:
yann@890
   691
yann@890
   692
 - a human-readable name, in lower case letters, with numbers as appropriate.
yann@890
   693
   The underscore is allowed; space and special characters are not (although
yann@890
   694
   they are internally replaced with underscores.
yann@890
   695
     Eg.: linux, bare-metal
yann@890
   696
 - a file in "config/kernel/", named after the kernel name, and suffixed with
yann@890
   697
   ".in".
yann@890
   698
     Eg.: config/kernel/linux.in, config/kernel/bare-metal.in
yann@901
   699
 - a file in "scripts/build/kernel/", named after the kernel name, and suffixed
yann@901
   700
   with ".sh".
yann@901
   701
     Eg.: scripts/build/kernel/linux.sh, scripts/build/kernel/bare-metal.sh
yann@890
   702
yann@890
   703
The kernel's ".in" file must contain:
yann@890
   704
 > an optional lines containing exactly "# EXPERIMENTAL", starting on the
yann@890
   705
   first column, and without any following space or other character.
yann@890
   706
   If this line is present, then this kernel is considered EXPERIMENTAL,
yann@890
   707
   and correct dependency on EXPERIMENTAL will be set.
yann@901
   708
yann@890
   709
 > the config option "KERNEL_%kernel_name%" (where %kernel_name% is to be
yann@890
   710
   replaced with the actual kernel name, with all special characters and
yann@890
   711
   spaces replaced by underscores).
yann@890
   712
   That config option must have *neither* a type, *nor* a prompt! Also, it can
yann@890
   713
   *not* depends on EXPERIMENTAL.
yann@890
   714
     Eg.: KERNEL_linux, KERNEL_bare_metal
yann@890
   715
   + mandatory:
yann@890
   716
       defines a (terse) help entry for this kernel.
yann@890
   717
       Eg.:
yann@890
   718
         config KERNEL_bare_metal
yann@890
   719
           help
yann@890
   720
             Build a compiler for use without any kernel.
yann@890
   721
   + optional:
yann@890
   722
       selects adequate associated config options.
yann@890
   723
       Eg.:
yann@890
   724
         config KERNEL_bare_metal
yann@890
   725
           select BARE_METAL
yann@890
   726
           help
yann@890
   727
             Build a compiler for use without any kernel.
yann@890
   728
yann@890
   729
 > other kernel specific options, at your discretion. Note however that, to
yann@890
   730
   avoid name-clashing, such options should be prefixed with
yann@890
   731
   "KERNEL_%kernel_name%", where %kernel_name% is again tp be replaced with
yann@890
   732
   the actual kernel name.
yann@890
   733
   (Note: due to historical reasons, and lack of time to clean up the code,
yann@890
   734
    I may have left some config options that do not completely conform to
yann@890
   735
    this, as the kernel name was written all upper case. However, the prefix
yann@890
   736
    is unique among kernels, and does not cause harm).
yann@890
   737
yann@901
   738
The kernel's ".sh" file API:
yann@901
   739
 > is a bash script fragment
yann@901
   740
yann@965
   741
 > defines the function CT_DoKernelTupleValues
yann@965
   742
   + see the architecture's CT_DoArchTupleValues, except for:
yann@965
   743
   + set the environment variable CT_TARGET_KERNEL, the kernel part of the
yann@965
   744
     target tuple
yann@965
   745
   + return value: ignored
yann@965
   746
yann@901
   747
 > defines the function "do_kernel_get":
yann@901
   748
   + parameters: none
yann@901
   749
   + environment:
yann@901
   750
      - all variables from the ".config" file.
yann@901
   751
   + return value: 0 for success, !0 for failure.
yann@901
   752
   + behavior: download the kernel's sources, and store the tarball into
yann@901
   753
     "${CT_TARBALLS_DIR}". To this end, a functions is available, that
yann@901
   754
     abstracts downloading tarballs:
yann@901
   755
     - CT_DoGet <tarball_base_name> <URL1 [URL...]>
yann@901
   756
       Eg.: CT_DoGet linux-2.6.26.5 ftp://ftp.kernel.org/pub/linux/kernel/v2.6
yann@901
   757
     Note: retrieving sources from svn, cvs, git and the likes is not supported
yann@901
   758
     by CT_DoGet. You'll have to do this by hand, as it is done for eglibc in
yann@901
   759
     "scripts/build/libc/eglibc.sh"
yann@901
   760
yann@901
   761
 > defines the function "do_kernel_extract":
yann@901
   762
   + parameters: none
yann@901
   763
   + environment:
yann@901
   764
      - all variables from the ".config" file,
yann@901
   765
   + return value: 0 for success, !0 for failure.
yann@901
   766
   + behavior: extract the kernel's tarball into "${CT_SRC_DIR}", and apply
yann@901
   767
     required patches. To this end, a function is available, that abstracts
yann@901
   768
     extracting tarballs:
yann@901
   769
     - CT_ExtractAndPatch <tarball_base_name>
yann@901
   770
       Eg.: CT_ExtractAndPatch linux-2.6.26.5
yann@901
   771
yann@901
   772
 > defines the function "do_kernel_headers":
yann@901
   773
   + parameters: none
yann@901
   774
   + environment:
yann@901
   775
      - all variables from the ".config" file,
yann@901
   776
   + return value: 0 for success, !0 for failure.
yann@901
   777
   + behavior: install the kernel headers (if any) in "${CT_SYSROOT_DIR}/usr/include"
yann@901
   778
yann@901
   779
 > defines any kernel-specific helper functions
yann@901
   780
   These functions, if any, must be prefixed with "do_kernel_%CT_KERNEL%_",
yann@901
   781
   where '%CT_KERNEL%' is to be replaced with the actual kernel name, to avoid
yann@901
   782
   any name-clashing.
yann@901
   783
yann@901
   784
You can have a look at "config/kernel/linux.in" and "scripts/build/kernel/linux.sh"
yann@903
   785
as an example of what a complex kernel description looks like.
yann@901
   786
yann@620
   787
Adding a new version of a component |
yann@476
   788
------------------------------------+
yann@476
   789
yann@476
   790
When a new component, such as the Linux kernel, gcc or any other is released,
yann@476
   791
adding the new version to crosstool-NG is quite easy. There is a script that
yann@476
   792
will do all that for you:
yann@1095
   793
  scripts/addToolVersion.sh
yann@476
   794
yann@476
   795
Run it with no option to get some help.
yann@381
   796
yann@203
   797
Build scripts |
yann@476
   798
--------------+
yann@203
   799
yann@203
   800
To Be Written later...