Saturday, December 20, 2008

View Android Source Code in Eclipse

Recently, I have been hacking around with Android a lot, and I think it's awesome.

Unfortunately, the Android SDK does not come with source code, and sometimes the docs are not so great, yet. Being an open source project, the Android sources are available online, but they are not very easy to access from Eclipse. For one, the source files are sometimes difficult to find as they are distributed over many individual Git repositories, and then I am not always online.

Integrating source files into Eclipse is usually quite easy, but the Android Eclipse plugin (ADT) does not allow to modify the Java Source Attachment of android.jar in the Android Library. Fortunately, you only have to find out what the default location for source code is. Eric Burke nicely describes how to find that out.

Long story short: Android source code needs to be placed in a "sources" subdirectory of the Android SDK. Here is what you need to do:

  1. Get the Android source code (install repo, repo init, repo sync)
  2. Move all Java sources into a "sources" subdirectory of the Android SDK

Step 2 sounds easier than it is. There are a lot of Java files in the Android sources, and they are sprinkled all over the place. Eclipse, however, needs the source files in a standard Java source directory structure: The path of a source file needs to match its package name. To simplify this task, I have written a short Python script that recursively searches for Java files and packs them into a ZIP file. Unpack that source file into a "sources" subdirectory of the Android SDK. Enjoy:

from __future__ import with_statement  # for Python < 2.6

import os
import re
import zipfile

# open a zip file
DST_FILE = 'sources.zip'
if os.path.exists(DST_FILE):
  print DST_FILE, "already exists"
  exit(1)
zip = zipfile.ZipFile(DST_FILE, 'w', zipfile.ZIP_DEFLATED)

# some files are duplicated, copy them only once
written = {}

# iterate over all Java files
for dir, subdirs, files in os.walk('.'):
  for file in files:
    if file.endswith('.java'):
      # search package name
      path = os.path.join(dir, file)
      with open(path) as f:
        for line in f:
          match = re.match(r'\s*package\s+([a-zA-Z0-9\._]+);', line)
          if match:
            # copy source into the zip file using the package as path
            zippath = match.group(1).replace('.', '/') + '/' + file
            if zippath not in written:
              written[zippath] = 1
              zip.write(path, zippath)
            break;
         
zip.close()

34 comments:

  1. NealMcBJan 3, 2009 04:06 PM
    Excellent! Thanks.
    The discussion at this link was also helpful.

    http://stuffthathappens.com/blog/2008/11/01/browsing-android-source-in-eclipse/
    ReplyDelete
  2. PeterJan 21, 2009 09:28 PM
    On windows, An easy way to just get a copy of the framework source is to install a windows version of git & ignore the google repo instructions.
    I used "Git Gui" and used their "Git Bash" shell program to navigate to a directory of my choice and then executed:

    "git clone git://android.git.kernel.org/platform/frameworks/base.git"

    You still need to move everything around to get into a nice format for eclipse...
    ReplyDelete
  3. edamFeb 8, 2009 05:29 AM
    It might be worth noting that if you follow the instructions on the google site to get the Android source code, you will be getting the latest source code. This will not match the code that comes with the SDK.

    To get the correct source code, when you init the repo, you should do it like this:

    $ repo init -u git://android.git.kernel.org/platform/manifest.git -b release-1.0
    ReplyDelete
  4. stanbApr 1, 2009 03:27 PM
    To browse just a single file, go to http://www.google.co.in/codesearch and search for: android calendar
    for example.
    ReplyDelete
  5. Gil MichlinApr 18, 2009 05:06 PM
    you must not include OUT directory
    or you will have SRC which is stub
    so add the following after os.walk:

    if dir=='.'
      if subdirs.count('out')>0:
        subdirs.remove('out')
    ReplyDelete
  6. secoifApr 27, 2009 10:42 AM
    Could someone please provide a link to download the 1.1 source files... poor internet connection rules 2gb downloads out for me so this would be really really helpful :) :)
    ReplyDelete
  7. poohtbearJul 28, 2009 11:25 PM
    Great post! great link as well, found it very useful.
    However i was unable to use repo init command, not that i got error message, it just takes too long! i got "receiving objects" to 91% and stopped there, and i left the process overnight...
    trying with git directly didn't do the trick most of the time either.One time (that was enough ) i've made it to pull out just the java sources and put them in sources.

    Anyone knows why it's soooooo slow ? is there another official mirror to the git ?
    ReplyDelete
  8. ssaidwhoOct 13, 2009 01:51 PM
    Thank you! The referenced blog and this script worked great for me.
    ReplyDelete
  9. idJan 4, 2010 11:57 PM
    perfect. thank you!
    ReplyDelete
  10. sosiouxmeJan 5, 2010 08:47 PM
    Thanks, incredibly useful.

    To edam's point - now there are a few more releases out, and they're not logically named release-1.1, release-1.5, etc.; the *tags* are but you can't get those with -b. Looking at the output of repo I'm thinking the branch names you want are cupcake, donut, or eclair, to debug with for your 1.5, 1.6, or 2.0 platforms respectively. Or do we want the -release versions of those?

    To secoif's point - I don't see any reason why, after I get the source all nicely located for a particular release, I couldn't zip it up and make it available on the web... will make a post if I get it all set...
    ReplyDelete
  11. ntraftJan 8, 2010 06:50 AM
    Does anyone know if Gil Michlin is correct about the OUT directories? I'm not familiar with Python and I'm not sure exactly where his code should be placed.
    ReplyDelete
  12. sosiouxmeJan 19, 2010 07:01 AM
    Those who don't have the connection to pull down 2GB of repo, check out my article for a summary and to download platform-specific 22MB source zips :-)

    http://android.opensourceror.org/2010/01/18/android-source/

    Let me know how it goes...
    ReplyDelete
  13. EneaJun 4, 2010 09:53 AM
    Awesome script.
    Thank you !
    ReplyDelete
  14. AlexJun 28, 2010 12:22 AM
    if dir=='.':
    if subdirs.count('out')>0:
    subdirs.remove('out')

    Note: the first line should have a : at the end or you get a python syntax error.

    He is correct; you have to exclude OUT directories or you will get stubs only. Copy/paste this script in a file 'SomeFilename.py' one level above your android source (/mydroid). From the command line you run 'python SomeFilename.py' (without quotes). This creates a zip file you then extract per the instructions above.
    ReplyDelete
  15. cdhulloAug 2, 2010 06:49 AM
    Unfortunately this doesn't work on eclipse 3.6 for OSX just wondering if anyone has a workaround? (works fine on 3.5 and in 3.6 on PC though)
    ReplyDelete
  16. wujek.srujekAug 28, 2010 02:53 AM
    I don't know if this has been posted yet, but it is not necessary to checkout all the projects. The project most devs are mostly interested in is 'platform/frameworks/base.git' which has the sources for android.jar. So, instead of checking out >2GB of sources for the kernel, tools and so on, you can just check out this project. I did it like this and it works:

    $> git clone git://android.git.kernel.org/platform/frameworks/base.git android

    Might take a few minutes, but it's definitely not 2GB.

    $> git tag

    You get a list of tags, so choose the tag you are interested in. I chose 'android-2.2_r1.1' as I want the FroYo sources.

    $> git checkout android-2.2_r1.1

    Now your working copy is reflecting the tag, not HEAD or whaterver else.
    What is left is just make use of the Python script.

    Go to the page: http://android.git.kernel.org. There you have a list of Android projects, and you can cherry pick what you want to download, for example there are the standard Android applications that were open sourced, the kernel, and so on.
    ReplyDelete
  17. Thomas DAOct 15, 2010 07:34 AM
    @wujek.srujek That is great!
    ReplyDelete
  18. ShreeDec 3, 2010 09:23 AM
    Do I have to unpack the files once I do the repo sync
    ReplyDelete
  19. Uwe GüntherJan 15, 2011 10:03 AM
    If you want to pack all tag based source (sugested by wujek.srujek) at once in a loop and if we assume the python script aboved is calle pack.py just fire up a simple bash loop like this:

    for I in $(git tag | grep android-[12]); do git checkout $I; python ../pack.py ; mv sources.zip ../$I-sources.zip; done
    ReplyDelete
  20. Uwe GüntherJan 15, 2011 10:10 AM
    Basically what you want to do in the bash is:

    git clone git://android.git.kernel.org/platform/frameworks/base.git android
    cp $WHEREVER/pack.py . #or simply create pack.py in the current dir
    cd android
    for I in $(git tag | grep android-[12]); do git checkout $I; python ../pack.py ; mv sources.zip ../$I-sources.zip; done
    cd ..
    ls -al *-sources.zip
    ReplyDelete
  21. jeetJan 19, 2011 10:52 PM
    hi all

    I have downloaded the complete source code for android in windows now want to build it on eclipse but i am unable open all pp together can any one help that how can i open complete source in android.
    ReplyDelete
  22. VolureMar 12, 2011 03:12 PM
    This is an excellent post. I have the script and have gotten the source code.
    If it was a pain in the behind until i found this post and to give back for the help you have, i have uploaded the files to dropbox
    The sources for sdk 9 and 10 (gingerbread):
    http://dl.dropbox.com/u/8034032/android/sources-9%20and%2010.zip
    The sources for sdk 8 (froyo):
    http://dl.dropbox.com/u/8034032/android/sources-8.zip

    I hope this helps some of the community with their development.

    It is so much easier to debug when you can walk the android code as well.

    Thanks again
    ReplyDelete
  23. VolureMar 12, 2011 03:13 PM
    oh yeah.
    Place these in their respective locations.

    /android-sdk/platforms/android-8/sources
    /android-sdk/platforms/android-9/sources
    /android-sdk/platforms/android-10/sources
    ReplyDelete
  24. bumbobwayMar 23, 2011 05:24 AM
    @Volure, Sources 9 and 10 are in one directory. How do I split them into their respective locations as you mentioned before?
    ReplyDelete
  25. PaulApr 4, 2011 12:06 PM
    Hi, I recently created a script that downloads every Android sources and creates a .jar archive for each version.
    You can download android sources .jars and/or get the script's code here :
    http://paul-wintzer.fr/?q=node/7
    ReplyDelete
  26. swingkidApr 9, 2011 06:55 AM
    This may help some what.
    but there is this tool called a decompiler.
    in Eclipse there is this plugin that i discovered awhile back
    called jd-eclipse which will attempt to decompile jars for you on the fly
    into source code.
    http://java.decompiler.free.fr/?q=node/276

    It you are runnig on a windows based platform and not the cutting edge eclipse, maybe 3.5 I thought it was a very useful and cool tool.
    Saves you time for having to hunt down and associate source code, just to get a feel for the flow of a program.
    ReplyDelete
  27. IannickApr 17, 2011 07:31 PM
    Hummm, I'm desperate...

    I'm on windows and I tried all that in that post... I have a LG Optimus One and I debug with it.

    Like Volure, I have a different structure the 1.5 ... I have
    /android-sdk/platforms/android-3
    /android-sdk/platforms/android-4
    /android-sdk/platforms/android-7
    ... 8,9,10 and 11

    I unzip the respective android.jar file into sources forder in each on these forlers.
    In my Eclipse, I set the source to the /android-sdk/platforms/android-3 (minimum)... But my phone runs on 2.2 (api level 8 ?)... I tied also to set the source to the /android-sdk/platforms/android-7 and nothing... oufff.... Someone have an idea ?
    ReplyDelete
  28. RyanMay 11, 2011 08:10 PM
    man, i'm in the same boat as Iannick. I'm on Ubuntu tho. did all the above, noted that my .jar was in the android-7 directory (from the Eclipse java viiew), but that directory had no "sources" directory. I created one, copied and unpacked the zip file there. Saw the files in Eclipse, but still got the "not found" error when running my app.

    I'm new to droid development, what information can I provide to help solve my problem?

    thanks
    ReplyDelete
  29. Struggling AuthorJun 9, 2011 06:23 AM
    @Volure
    Awesomeness! Thanks so much for the zip files.
    ReplyDelete
  30. Albert E.Jun 30, 2011 11:55 PM
    Just want to say thanks for the script. I tried to create my own zip file and failed every time. Downloaded the source twice even to make sure I had everything. Ran your script and Voila!!, working db driven HC app. Thanks!!
    ReplyDelete
  31. CodeboldAug 29, 2011 03:43 PM
    Thanks for the script! Saved my night...
    Greetings from Passau!
    ReplyDelete
  32. mr. AndersonDec 22, 2011 07:00 AM
    I have done that. And all I could do is step through comments. I think it is the lin enumber mismatch.
    ReplyDelete
  33. Kiran GundaDec 25, 2011 09:07 PM
    If you are using eclipse, then it now gives a better and very simple way to add the source code.

    1. Go to Window -> Android SDK Manager
    2. It automatically shows the new packages/updates to existing packages. Packages are grouped by API Level.
    3. Go to the required API - Level group and you will find an option "Sources for Android SDK" checkbox. Select this check box and click on 'Install Packages' button. You may want to un-select the packages that you don't need.
    4. Now the source code is downloaded to //sources/ folder.
    5. Open any Android class in the Eclipse editor and it now asks for the source path.. Give the source path as //sources/.

    Kiran
    ReplyDelete