Saturday, December 20, 2008

View Android Source Code in Eclipse

Recently, I have been hacking around with Android a lot, and I think it's awesome.

Unfortunately, the Android SDK does not come with source code, and sometimes the docs are not so great, yet. Being an open source project, the Android sources are available online, but they are not very easy to access from Eclipse. For one, the source files are sometimes difficult to find as they are distributed over many individual Git repositories, and then I am not always online.

Integrating source files into Eclipse is usually quite easy, but the Android Eclipse plugin (ADT) does not allow to modify the Java Source Attachment of android.jar in the Android Library. Fortunately, you only have to find out what the default location for source code is. Eric Burke nicely describes how to find that out.

Long story short: Android source code needs to be placed in a "sources" subdirectory of the Android SDK. Here is what you need to do:

  1. Get the Android source code (install repo, repo init, repo sync)
  2. Move all Java sources into a "sources" subdirectory of the Android SDK

Step 2 sounds easier than it is. There are a lot of Java files in the Android sources, and they are sprinkled all over the place. Eclipse, however, needs the source files in a standard Java source directory structure: The path of a source file needs to match its package name. To simplify this task, I have written a short Python script that recursively searches for Java files and packs them into a ZIP file. Unpack that source file into a "sources" subdirectory of the Android SDK. Enjoy:

from __future__ import with_statement  # for Python < 2.6

import os
import re
import zipfile

# open a zip file
DST_FILE = 'sources.zip'
if os.path.exists(DST_FILE):
  print DST_FILE, "already exists"
  exit(1)
zip = zipfile.ZipFile(DST_FILE, 'w', zipfile.ZIP_DEFLATED)

# some files are duplicated, copy them only once
written = {}

# iterate over all Java files
for dir, subdirs, files in os.walk('.'):
  for file in files:
    if file.endswith('.java'):
      # search package name
      path = os.path.join(dir, file)
      with open(path) as f:
        for line in f:
          match = re.match(r'\s*package\s+([a-zA-Z0-9\._]+);', line)
          if match:
            # copy source into the zip file using the package as path
            zippath = match.group(1).replace('.', '/') + '/' + file
            if zippath not in written:
              written[zippath] = 1
              zip.write(path, zippath)
            break;
         
zip.close()

45 comments:

  1. Excellent! Thanks.
    The discussion at this link was also helpful.

    http://stuffthathappens.com/blog/2008/11/01/browsing-android-source-in-eclipse/

    ReplyDelete
  2. On windows, An easy way to just get a copy of the framework source is to install a windows version of git & ignore the google repo instructions.
    I used "Git Gui" and used their "Git Bash" shell program to navigate to a directory of my choice and then executed:

    "git clone git://android.git.kernel.org/platform/frameworks/base.git"

    You still need to move everything around to get into a nice format for eclipse...

    ReplyDelete
  3. It might be worth noting that if you follow the instructions on the google site to get the Android source code, you will be getting the latest source code. This will not match the code that comes with the SDK.

    To get the correct source code, when you init the repo, you should do it like this:

    $ repo init -u git://android.git.kernel.org/platform/manifest.git -b release-1.0

    ReplyDelete
  4. To browse just a single file, go to http://www.google.co.in/codesearch and search for: android calendar
    for example.

    ReplyDelete
  5. you must not include OUT directory
    or you will have SRC which is stub
    so add the following after os.walk:

    if dir=='.'
      if subdirs.count('out')>0:
        subdirs.remove('out')

    ReplyDelete
  6. Could someone please provide a link to download the 1.1 source files... poor internet connection rules 2gb downloads out for me so this would be really really helpful :) :)

    ReplyDelete
  7. Great post! great link as well, found it very useful.
    However i was unable to use repo init command, not that i got error message, it just takes too long! i got "receiving objects" to 91% and stopped there, and i left the process overnight...
    trying with git directly didn't do the trick most of the time either.One time (that was enough ) i've made it to pull out just the java sources and put them in sources.

    Anyone knows why it's soooooo slow ? is there another official mirror to the git ?

    ReplyDelete
  8. Thank you! The referenced blog and this script worked great for me.

    ReplyDelete
  9. Thanks, incredibly useful.

    To edam's point - now there are a few more releases out, and they're not logically named release-1.1, release-1.5, etc.; the *tags* are but you can't get those with -b. Looking at the output of repo I'm thinking the branch names you want are cupcake, donut, or eclair, to debug with for your 1.5, 1.6, or 2.0 platforms respectively. Or do we want the -release versions of those?

    To secoif's point - I don't see any reason why, after I get the source all nicely located for a particular release, I couldn't zip it up and make it available on the web... will make a post if I get it all set...

    ReplyDelete
  10. Does anyone know if Gil Michlin is correct about the OUT directories? I'm not familiar with Python and I'm not sure exactly where his code should be placed.

    ReplyDelete
  11. Those who don't have the connection to pull down 2GB of repo, check out my article for a summary and to download platform-specific 22MB source zips :-)

    http://android.opensourceror.org/2010/01/18/android-source/

    Let me know how it goes...

    ReplyDelete
  12. Awesome script.
    Thank you !

    ReplyDelete
  13. if dir=='.':
    if subdirs.count('out')>0:
    subdirs.remove('out')

    Note: the first line should have a : at the end or you get a python syntax error.

    He is correct; you have to exclude OUT directories or you will get stubs only. Copy/paste this script in a file 'SomeFilename.py' one level above your android source (/mydroid). From the command line you run 'python SomeFilename.py' (without quotes). This creates a zip file you then extract per the instructions above.

    ReplyDelete
  14. Unfortunately this doesn't work on eclipse 3.6 for OSX just wondering if anyone has a workaround? (works fine on 3.5 and in 3.6 on PC though)

    ReplyDelete
  15. I don't know if this has been posted yet, but it is not necessary to checkout all the projects. The project most devs are mostly interested in is 'platform/frameworks/base.git' which has the sources for android.jar. So, instead of checking out >2GB of sources for the kernel, tools and so on, you can just check out this project. I did it like this and it works:

    $> git clone git://android.git.kernel.org/platform/frameworks/base.git android

    Might take a few minutes, but it's definitely not 2GB.

    $> git tag

    You get a list of tags, so choose the tag you are interested in. I chose 'android-2.2_r1.1' as I want the FroYo sources.

    $> git checkout android-2.2_r1.1

    Now your working copy is reflecting the tag, not HEAD or whaterver else.
    What is left is just make use of the Python script.

    Go to the page: http://android.git.kernel.org. There you have a list of Android projects, and you can cherry pick what you want to download, for example there are the standard Android applications that were open sourced, the kernel, and so on.

    ReplyDelete
  16. Do I have to unpack the files once I do the repo sync

    ReplyDelete
  17. If you want to pack all tag based source (sugested by wujek.srujek) at once in a loop and if we assume the python script aboved is calle pack.py just fire up a simple bash loop like this:

    for I in $(git tag | grep android-[12]); do git checkout $I; python ../pack.py ; mv sources.zip ../$I-sources.zip; done

    ReplyDelete
  18. Basically what you want to do in the bash is:

    git clone git://android.git.kernel.org/platform/frameworks/base.git android
    cp $WHEREVER/pack.py . #or simply create pack.py in the current dir
    cd android
    for I in $(git tag | grep android-[12]); do git checkout $I; python ../pack.py ; mv sources.zip ../$I-sources.zip; done
    cd ..
    ls -al *-sources.zip

    ReplyDelete
  19. hi all

    I have downloaded the complete source code for android in windows now want to build it on eclipse but i am unable open all pp together can any one help that how can i open complete source in android.

    ReplyDelete
  20. This is an excellent post. I have the script and have gotten the source code.
    If it was a pain in the behind until i found this post and to give back for the help you have, i have uploaded the files to dropbox
    The sources for sdk 9 and 10 (gingerbread):
    http://dl.dropbox.com/u/8034032/android/sources-9%20and%2010.zip
    The sources for sdk 8 (froyo):
    http://dl.dropbox.com/u/8034032/android/sources-8.zip

    I hope this helps some of the community with their development.

    It is so much easier to debug when you can walk the android code as well.

    Thanks again

    ReplyDelete
  21. oh yeah.
    Place these in their respective locations.

    /android-sdk/platforms/android-8/sources
    /android-sdk/platforms/android-9/sources
    /android-sdk/platforms/android-10/sources

    ReplyDelete
  22. @Volure, Sources 9 and 10 are in one directory. How do I split them into their respective locations as you mentioned before?

    ReplyDelete
  23. Hi, I recently created a script that downloads every Android sources and creates a .jar archive for each version.
    You can download android sources .jars and/or get the script's code here :
    http://paul-wintzer.fr/?q=node/7

    ReplyDelete
  24. This may help some what.
    but there is this tool called a decompiler.
    in Eclipse there is this plugin that i discovered awhile back
    called jd-eclipse which will attempt to decompile jars for you on the fly
    into source code.
    http://java.decompiler.free.fr/?q=node/276

    It you are runnig on a windows based platform and not the cutting edge eclipse, maybe 3.5 I thought it was a very useful and cool tool.
    Saves you time for having to hunt down and associate source code, just to get a feel for the flow of a program.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  25. Hummm, I'm desperate...

    I'm on windows and I tried all that in that post... I have a LG Optimus One and I debug with it.

    Like Volure, I have a different structure the 1.5 ... I have
    /android-sdk/platforms/android-3
    /android-sdk/platforms/android-4
    /android-sdk/platforms/android-7
    ... 8,9,10 and 11

    I unzip the respective android.jar file into sources forder in each on these forlers.
    In my Eclipse, I set the source to the /android-sdk/platforms/android-3 (minimum)... But my phone runs on 2.2 (api level 8 ?)... I tied also to set the source to the /android-sdk/platforms/android-7 and nothing... oufff.... Someone have an idea ?

    ReplyDelete
  26. man, i'm in the same boat as Iannick. I'm on Ubuntu tho. did all the above, noted that my .jar was in the android-7 directory (from the Eclipse java viiew), but that directory had no "sources" directory. I created one, copied and unpacked the zip file there. Saw the files in Eclipse, but still got the "not found" error when running my app.

    I'm new to droid development, what information can I provide to help solve my problem?

    thanks

    ReplyDelete
  27. @Volure
    Awesomeness! Thanks so much for the zip files.

    ReplyDelete
  28. Just want to say thanks for the script. I tried to create my own zip file and failed every time. Downloaded the source twice even to make sure I had everything. Ran your script and Voila!!, working db driven HC app. Thanks!!

    ReplyDelete
  29. Thanks for the script! Saved my night...
    Greetings from Passau!

    ReplyDelete
  30. I have done that. And all I could do is step through comments. I think it is the lin enumber mismatch.

    ReplyDelete
  31. If you are using eclipse, then it now gives a better and very simple way to add the source code.

    1. Go to Window -> Android SDK Manager
    2. It automatically shows the new packages/updates to existing packages. Packages are grouped by API Level.
    3. Go to the required API - Level group and you will find an option "Sources for Android SDK" checkbox. Select this check box and click on 'Install Packages' button. You may want to un-select the packages that you don't need.
    4. Now the source code is downloaded to //sources/ folder.
    5. Open any Android class in the Eclipse editor and it now asks for the source path.. Give the source path as //sources/.

    Kiran

    ReplyDelete
    Replies
    1. Thanks! You made my day! I already HAD the sources downloaded. It simply wasn't connected in Eclipse. This is so much easier than reading through all the pages for getting the whole repository with repo...

      Delete
    2. Nice! Thank you! I also had it downloaded already, just needed to provide the source: I am new to this so for me all I did was right click on "Activity{" in one of my android projects> open declaration> then provided the source "C:\Program Files (x86)\Android\android-sdk\sources" and that was it.

      Delete
    3. This comment has been removed by the author.

      Delete
  32. Hi,

    thanks for the tutorial.

    I can now browse the android source code, but not edit it. How can I edit the source code? Thanks!

    ReplyDelete
    Replies
    1. hi. would you explain me more on that? im totally new in developing android apps. really need a helping hand. Thanks!

      Delete
  33. Thank you, it worked perfectly.
    I can also confirm that only packaging the /frameworks/base/ directory with the python script is sufficient.

    @Kiran Gunda: Sources in the Android SDK Manager only seem to be available for API Level 14 and 15. If you need Level 10 like me, this guide helps.

    ReplyDelete
  34. hi. im new in developing android apps. recently i need to work on a android project assignment. i cant really understand this. Is this means that we hacked into available android apps and get the source code? any reply would really valuable for me. thanks!

    ReplyDelete
  35. You can try to visit the Android Source Code Online service.
    There are latest Linux kernel source code and the update today Android framework source code.
    Support cross reference interface to search and browsing source code in the website interface.

    Android/Linux Source Code Cross Reference

    http://hala01.com/

    ReplyDelete
  36. Everything worked perfectly == I could download source for gingerbread - and made sources.zip - and extracted it - but I can't use/build it in Eclipse Juno - properly - please help ...

    ReplyDelete