Resource organization for building firmware

Ps you may want to look at the new build tools as well http://ardupilot.org/dev/docs/building-the-code.html

Make is being deprecated

hey martinandriener

I guess I might help you a bit, I’m on the same boat called windows. I’m not with Rover but Copter, and am on Win7 and not WinXP, but my procedure is a s follows

  1. first, and most importantly, you should know these wiki pages:
  1. download pixhawk_toolchain_installer_latest.exe and run it
    this is actually quite a nice piece, since it installs everything but doesn’t hurt any other installation on your PC, so, if you have e.g. an arm-eabi somewhere else, or python, or whatever, this is not affected and no conflict happens !
    there is a bit of a downside in that it now makes you installing all drivers, and doesn’t give you the option to skip that part as in older versions, which is nasty since it takes some time, and I never had found these drivers to work out for me (e.g. make-upload doesn’t work), however, I also never found these drivers to hurt, so, no need to worry here
    there is also a bit of downside in that it is totally unclear which version the pixhawk_toolchain_installer is, so you essentially have to reinstall it often
    otherwise it’s really a nice piece, since totally dependence free
    so, JUST RUN IT, say always yes to everything, and you get all you need

  2. get the code
    this step depends on which tools you’re using, I’m using Github Desktop, and not Git for Windows as mentioned in the wiki. Github Desktop makes it quite easy, but needs maybe a bit of getting used to it. Anyway, with your git-based tool, clone the master repository, and check out the branch you want (master, 3.4., or whatever), you should then have a github folder on your drive with an ardupilot subfolder.
    As regards downloading the zip: In my “early” days I also did it by downloading the .zip, and it worked fine back then, but I would NOT recommend that anymore! The reason is that they are now heavily using git submodules (or however that is correctly called in git-language), and to get the code of those is going to be a real pain! This is where I stumbled and which made me using a git-based tool to get the firmware folders from the git repository.
    So, DON’T WASTE YOUR TIME with zips, you’re much better off using one of the two mentioned git things.

  3. get the submodules
    this step also may depend on the git tool you’re using
    Github desktop does it all for you automatically, i.e. you don’t have to do this step, for other tools I don’t know.
    With Github Desktop it would manually go as this:
    open GitShell, browse to the git firmware folder, e.g. github\ardupilot, and enter git submodule update --init --recursive
    your git tool should also offer you a command line thingy, where to enter git submodule update --init --recursive

  4. copy and “initialize” the folders
    I’m not developing using git, and want code organized “my way”, so what I do is to copy the ardupilot folder to somewhere else and rename it, let’s say blabla/xpilot. This will break the git, and it will make the px4 toolchain to not work. So, in the blabla/xpilot folder, what I do is this:
    delete the .git and .github subfolders
    open GitShell, and browse to the blabla/xpilot folder
    run git init
    run git add *
    run git commit -m "any comment you like"
    this corrects the git

  5. setup Eclipse
    in step (2) you should have gotten a PX4 Toolchain entry in the Start Menu, go there, and run PX4 Eclipse
    when, follow the f… manual: http://ardupilot.org/dev/docs/editing-the-code-with-eclipse.html#editing-the-code-with-eclipse
    I do the “Creating the Project from scratch” thing
    the only thing I do differently is the first step, where you set up the new project. Instead of the mentioned name&folder, I enter whatever name I find convenient, and select the “…\blabla\xpilot” subfolder, and NOT the vehicle subfolder. The advantage is that I then also can easily browse to e.g. the libraries, look into the code of other vehicles, and so on.
    Note that in the later step, where you enter the build folder, you MUST use the vehicle subfolder, as told in the wiki.

That’s it. This was a lot of text, my memo sheet looks much shorter, as this
clone into GitHub
open Git Shell, git submodule update --init --recursive
copy to desired folder, rename
delete .git and .github
open Git Shell, git init, git add *, git commit -m "whateveryouwant"
setup Eclipse: http://ardupilot.org/dev/docs/editing-the-code-with-eclipse.html#editing-the-code-with-eclipse

This at least works fine for me.
Have fun and enjoy it for as long as it lasts, my extrapolation is that Windows folks won’t be able to compile ArduPilot in a near future.

Olli

I see there are a lot of other useful answers now, but just on one point.
arm-none-eabi-g++ is the compiler that is used to build the pxf version of ardupilot. rather than your system g++
There is no collision I think since it is invoked by the build system as arm-none-eabi-g++ rather than g++

It also sound like there are issues the way you have your tree set up, since I think ardupilot should be the ‘top level’

Anyway I hope you get this sorted. It is possible just a case of figuring out what is the problem :slight_smile:

Windows XP

I really don’t think you are going to have success on XP, you may, but Windows7 is really a minimum in this case.

@billb, @skyscraper, and especially @olliw42, I greatly appreciate your thoughts on this and the time that you took to compose helpful responses.

As you get my drift, I’m as much interested in understanding more deeply how the build (compiling/linking) process works, even how it could alternatively work, as I am interested in just getting something to work. (One point I’m specifically interested in clearing up is whether the “ardupilot” directory belongs inside the “Firwmare” directory or vice-versa.)

I’m not quitting on Windows XP just yet as I reckon that with MinGW32 and all the tools that are part of the v16 pixhawk toolchain one ought to be able to build the source code if one fully understands all the linking. But I’m convinced to get a linux machine going as well.

It won’t be the first regrouping/digression for me; About a month and a half in September and October were devoted to studying Lippman, Lajoie and Moo C++ Primer along with Stroustrup’s C++ bible, since my previous programming experience had been in C. Maybe I also need to take the next step in remedial education and study more on the general topic of code-building.

Whew, I so regret not having gotten into the ardupilot game about 2009. I’m seriously considering rewriting Pixhawk-platform-specific code in a way that I can manage, especially since I aim to use true heading data from gyro (actually GPS-based), input to serial port, rather than using heading derived from magnetometer on I2C bus. But at the end of the day I don’t want to lose MAVlink; I find Mission Planner to be an extremely useful development tool.

I don’t aim to reinvent wheels, but I sense the need (at least with my limited mental resources) to work with a less universal code base, and probably not within the playground of developers who are really in a different league than I.

Anyway, I’ll let you know how it goes as I follow up on good suggestions.

Cheers!
Pete

To understand the build process for px4, you could try looking at and working on https://github.com/PX4/Firmware . That is the PX4 library part of the px4 ArduPilot build and can be built separately. (Note that the ArduPilot version may not be compatible with the official px4 , since I think various patches are added). If you watch the build of that it should be familiar to the px4 Ardupilot build.

The other great way to understand the buils process is to capture the output to a file. There is a verbose option somewhere in the makefile which shows more of the actual compiler output, which by default is suppressed.
From memory, if you add a VERBOSE=True or similar to your command line (or just hack the makefile) it will show more output.
see
https://github.com/ArduPilot/ardupilot/blob/master/mk/sketch_sources.mk#L85

Re; learning C++. I started from Bjarne Stroustrup’s book and to me it was like coming home. C++ is my number 1 language. I think it is worth persevering with, but I guess I am biased.

Hi Pete,

Windows is a complicated developer environment but, as the lead dev of Copter uses it, we try to support it the best we can.
My recommendation is to install the latest Windows Pixhawk Toolchain and use a Git tool (possibly GitHub for Windows) to clone the ArduPilot repository. I think that using zips of the repository isn’t well supported at this time.
I would also clone the repository to a path outside the path of the toolchain.

If you need more help you may also go to our Gitter chat (http://gitter.im/ArduPilot/ardupilot). Interaction in real-time sometimes is more helpful.

I did succeed in building (unmodified) ardupilot source, essentially following simple suggestions from @gmorph in a more specific arduboat discussion thread about alternative heading input that I started quite a while back.

In summary, the essential steps for me, presently in a Windows XP OS, were:

  • Install Pixhawk Toolchain for Windows;
  • Run the PX4 Software Download (which does not fetch ardupilot per se);
  • In MINGW32 console window (“PX4 Console” of PX4 Toolchain), cd to my c:/pixhawk_toolchain/Firmware directory and do

git clone https://github.com/ArduPilot/ardupilot

  • Then cd to the ardupilot directory (my c:/pixhawk_toolchain/Firmware/ardupilot) and do

git submodule update --init --recursive

  • And then finally I was able to cd to the vehicle directory (my c:/pixhawk_toolchain/Firmware/ardupilot/APMrover2) and do

make px4-v2

…which resulted in the desired APMrover2-v2.px4 file in the same directory, which I successfully uploaded to the Pixhawk via Mission Planner (“Custom Firmware”) and ran OK.

There were some apparent errors reported during the build, for example

fatal: not a git repository: c:/pixhawk_toolchain/Firmware/ardupilot/modules/PX4nuttX/nuttx//…/…/.git/modules/modules/PX4NuttX

…which apparently did not preclude ultimately building the APMrover2-v2.px4 file. Incidentally, I copied the above carefully, and and don’t even understand the significance of the occurrence of “//”, nor of the “…/…/” …are we referring to a directory two levels back up at that point in the path??

The victory is a bit hollow for me since as yet I lack understanding of the “how” and “why” and “what else might” about this working, but needless to say I’m glad to keep going forward, with much gratitude to you all for help and insight.

@skyscraper, I think I kind of share your sense, albeit with less experience, about C++ in general and about Bjarne Stroustrup’s book in particular. (I have “The C++ Programming Language” 3rd edition, 1997.) I think he’s got insight to offer beyond just programming, and I enjoy the abstractions which I can relate to other areas of interest.

Best regards to all,
Pete

Perhaps to conclude this thread, this is meant to offer my own explicit answer to the question that I posed regarding the organization, in file hierarchical structure, of resources for building ardupilot code.

To recap, I was working in Windows environments, first XP and then Windows 7, and carried the process through to successful build of APMrover2 vehicle executable by the four steps:

  • Install pixhawk toolchain (obtained from http://firmware.us.ardupilot.org/Tools/PX4-tools/);
    This created “drivers”, “eclipse”, “Python27”, “toolchain”, and “tools” directories under C:/pixhawk_toolchain/. The tree at this point is like this.

  • Run “PX4 Software Download” part of pixhawk toolchain;
    This adds “Bootloader” and “Firmware” directories to c:/pixhawk_toolchain/. At this point c:/pixhawk_toolchain/Firmware contains directories “.git”, “cmake”, “Debug”, “Documentation”, “Images”, “integrationtests”, “launch”, “mavlink”, “misc”, “msg”, “NuttX”, “nuttx-configs”, “posix-configs”, “ROMFS”, “src”, “test_data”, “Tools” (not to be confused with “c:/pixhawk_toolchain/tools”), and “unittests”. It does not yet contain “ardupilot”. Except that hidden files and folders are not shown, the tree at this point is like this.

  • Start MINGW32 console window by the “PX4 Console Window” part of pixhawk toolchain, and after “cd Firmware”, do “git clone https://github.com/ArduPilot/ardupilot”;
    This adds the “ardupilot” directory to c:/pixhawk_toolchain/Firmware/. The seven module directories “gbenchmark”, “gtest”, “mavlink”, “PX4Firmware”, “PX4NuttX”, “uavcan” and “waf” are created under c:/pixhawk_toolchain/Firmware/ardupilot/modules/, but they are empty at this point. The tree at this point looks like this.

  • In MINGW32 console, after “cd ardupilot”, do "git submodule update --init --recursive;
    This fills the above-mentioned seven module directories, registering not only those seven modules but also Tools/gencpp, Tools/genmsg (the “Tools” directory for these two being c:/pixhawk_toolchain/Firmware/ardupilot/mk/PX4/Tools, not to be confused with either “tools” or “Tools” directories mentioned in step 2), mavlink/include/mavlink/v1.0 (that is, c:/pixhawk_toolchain/Firmware/mavlink/include/mavlink/v1.0, not down the c:/pixhawk_toolchain/Firmware/arupilot/modules/mavlink path), src/lib/DriverFramework, src/lib/ecl, src/lib/matrix, and src/modules/uavcan/libuavcan (all down the c:/pixhawk_toolchain/Firmware/src path). The tree at this point looks like this.

  • In MINGW32 console, after “cd APMrover2”, do “make px4-v2” (or initiate a build within eclipse);
    This produces the ~700kB “APMrover2-v2.px4” firmware file in the same vehicle directory.

I concede that it’s probably better not to concern oneself with such details, and that a snapshot account of resource organization like this is of dubious value in the ephemeral world of software development and repository management; but I thought it might be helpful by way of specific illustration, to one starting out, perhaps with the innocent expectation that code-building should be a fairly straight-forward two-step proposition of gathering files of source code into one place and then compiling it.

Cheers,
Pete

That step is not needed. That is actually downloading another flight controller software that you don’t need if you are using ArduPilot.

@OXINARF, Thank you very much for this clarification, Francisco. I just proved for myself that the “PX4 Software Download” step was extraneous; starting again from scratch, I successfully downloaded all necessary source from MINGW32 console by “git clone https://ArduPilot/ardupilot” followed by cd ardupilot and “git submodule update --init --recursive”, and then successfully built APMrover2-v2.px4 firmware by “make px4-v2” from the APMrover2 directory.

Cheers,
Pete

1 Like

Apologies if this question is too basic, but I think this thread that I started over a month ago got moved.

You all motivated me to get going with “development” using Linux, not just Windows; I finally got lubuntu 16.10 going on an old Thinkpad T-40. Anyway, attempting to build APMrover2 by make (following “Building Ardupilot for Pixhaw/PX4 on Linux Using Make”, even if make is being deprecated), there is this error, which I’m not adept enough to interpret.

If an expert eye can help me learn a bit, I’d once again be most grateful!

I suspect by invoking make by itself your trying to build the APM target which isn’t supported any more which is why its failing. You want to use waf. I’m going to assuming your building for a pixhawk. From the ArduPilot directory run

./modules/waf/waf-light configure --board px4-v2
./modules/waf/waf-light build --target bin/ardurover

That should work.

Thanks, Grant.

Thank you, Grant,

Actually, I should have said, from the ardupilot/APMrover2 directory, I invoked

make px4-v2

just as I had done when working on a Windows machine in a MinGW32 console a few weeks ago, to successfully build APMrover2.px4. But on the Linux machine it fails with the errors indicated by the link I gave.

Anyway, I tried the waf approach suggested and “configure” (invoked exactly as you suggested) produces the errors

Checking for header endian.h : not found
Checking for header byteswap.h : not found
Checking for program ‘cmake’ : not found
Could not find the program [‘cmake’]

Your missing some dependencies. Firstly make sure your repository is up to date (I assume its already setup and you have run “git submodule init”. From the ardupilot directory run:

git pull --rebase
git submodule update

Make sure you have read through all this

http://ardupilot.org/dev/docs/building-px4-for-linux-with-make.html#building-px4-for-linux-with-make

and you have run the pre-reqs installation script.

When using waf this has the instructions.

Thanks, Grant.

@gmorph, with great thanks to you, I had success building ardupilot code in a Linux environment, both APMrover2-v2.px4 using make, and ardurover.px4 using waf.

At the end of the day I think the critical step for me was to do Tools/scripts/install-prereqs-ubuntu.sh -y from ~/ardupilot; previously, though I tried to follow the “Building PX4 for Linux with Make” documentation very carefully, my notes indicate that I ran install-prereqs-ubuntu.sh -y from my ~/ardupilot/Tools/scripts directory.

Incidentally, this step took almost an hour with my ~8MB/min internet connection.

Also, for practice or proof, from ~/ardupilot I successfully did ./modules/waf/waf-light rover --upload to install the firmware on Pixhawk (rather than using Mission Planner as I had in the past).

Following your lead, for both building and uploading, I used waf-light in the ardupilot/modules/waf directory instead of waf at the top of the ardupilot directory as implicitly suggested in the github ArduPilot/ardupilot/blob/master/BUILD/md documentation to which you refer, without fathoming the possible differences in effect.

Anyway, kind thanks for your responsiveness and motivation!
Pete

You all motivated me to get going with “development” using Linux, not just Windows; I finally got lubuntu 16.10 going on an old Thinkpad T-40. Anyway, attempting to build APMrover2 by make (following “Building Ardupilot for Pixhaw/PX4 on Linux Using Make”, even if make is being deprecated), there is this error,
which I’m not adept enough to interpret.

Did the “git submodule update --init --recursive” return any errors?
Failure to check out the mavlink source has led to this sort of error…

Grant’s advice to use waf is solid.

@peterbarker,

To answer lately: the account of this saga was split between this thread and an ArduBoat discussion thread. I did ultimately succeed in building APMrover2.px4 both with “make” and with “waf”, on Windows XP, Windows 7, and Linux (lubuntu). “git submodule update --init --recursive” did return errors:

  • failed to recurse into submodule path ‘ibuavcan/dsdl_compiler/pyuavcan’
  • failed to recurse into submodule path ‘src/modules/uavcan/libuavcan’
  • failed to recurse into submodule path ‘modules/px4firmware’

…but the build appears to have succeeded (I uploaded and ran the resulting APMrover2-v2.px4).

Roger on Grant’s advice to use waf being solid. I proved that both in the Windows and Linux environments. Not to belabor my inquiry process, but I was just as interested in questions of “what else might work?” and “why?” or “why not?” as in the question of “how can I get this to work?” and “what is the latest mode (fashion)?”!

Cheers,
Pete

Pete,

I realize this is a bit late but I was wondering if you ever were able to get the code compiled using Eclipse and XP? I’ve spent way too much time trying to get my Windows 10 Eclipse setup to spit out a ArduPlane file. I’m getting all sorts of errors that it can’t find the FlightMode enums even though I’ve added the file locations in the Includes path.

Thanks,
Richard

Hi Richard,

Had to review my notes! Hard to remember anything since my country had its stroke last November…

No, unless I’ve forgotten, I was never able to compile APMrover2 px4-v2 using Eclipse, but did succeed from command window using both “make” and “waf”, on the three different platforms noted in #23 of this thread.

By the way, the reference link in that post above has a typo ("." following the final “10”); I touched on my own difficulties with Eclipse in that thread.

Sadly, I gave up on Eclipse. I thought it might help me to get a better handle on the maze of ArduPilot code, by means of function references and so forth, but it always seemed almost as lost as I.

Code organization not being the primary object of my interest, and finding that even ArduPilot experts don’t easily implement what seem to me to be very basic I/O changes, I’m drifting away from the unwieldy universality of ArduPilot in favour of the more rewarding and fun effort of developing my own leaner code for my specific ArduBoat (that might even run on an Arduino!).

Best to you,
Pete