Being early adopters has its downfalls for sure; some of which usually include system instability, a lack of published tools, and proper or extensive documentation. I'll try to help address the latter in this short tutorial.
Configuring Google Storage for App Engine
This problem has come up a couple of times in the Google Storage for Developers group. After looking at the Python Library Docs for GS, the biggest hurdle looks like getting the boto library properly configured. Once boto is set up, you should be able to run all of the sample code provided by Google. Lets get you there.
1 2 3 4 5 6 7
# Bring in the boto library import boto # Load the configuration variables from your .boto file config = boto.config """ FAIL. """
As you can see by my fail comment, App Engine has no way to access your ~/.boto file where your Secret Key is located. We can solve this by diving into boto.config. When the boto configuration is initialized, It checks for the initial file and tries to parse it and load it into memory by using add_section(section_name) and set(section_name,key,value). Knowing this, you're able to call those functions yourself after loading the boto lib and it runs flawlessly. It only took a little bit of hacking around to find a solution, but it works great for me. I'd love to hear your experience, or how you got it rolling differently.
pay close attention to the 3 lines following config = boto.config
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
"""" Python App Engine + Google Storage. Based heavily from examples given by Google Instructions: just rip the boto lib from the gsutil project, drop it in your python App Engine project. @author: Adam Thurlow <Thurloat> """" import boto from google.appengine.ext import webapp config = boto.config config.add_section('Credentials') config.set('Credentials', 'gs_access_key_id', 'YOURACCESSKEY') config.set('Credentials', 'gs_secret_access_key', 'YOURSECRETKEY') FLAGS = ['CREATE', 'LIST_BUCKETS', 'LIST_CONTENTS','DOWNLOAD_FILE'] DOWNLOAD_PATH = '/path/to/file.ext' DOWNLOAD_FILENAME = 'file.ext' BUCKET_NAME = 'testbucket' class DownloadHandler(webapp.RequestHandler): def get(self): srow = self.response.out.write if 'DOWNLOAD_FILE' in FLAGS: uri = boto.storage_uri("%s/%s" % (BUCKET_NAME, DOWNLOAD_PATH), 'gs') self.response.headers['Content-Type'] = 'application/octet-stream' self.response.headers.add_header('content-disposition', 'attachment', filename = DOWNLOAD_FILENAME) srow(uri.get_contents_as_string()) class MainHandler(webapp.RequestHandler): def get(self): srow = self.response.out.write srow("<h4> Testing the Google Storage API </h4>") if 'CREATE' in FLAGS: srow("<h2>Creating Bucket</h2>") uri = boto.storage_uri(BUCKET_NAME, 'gs') uri.create_bucket() if 'LIST_BUCKETS' in FLAGS: srow("<h2>Listing Buckets</h2>") uri = boto.storage_uri('', 'gs') buckets = uri.get_all_buckets(); for bucket in buckets: self.response.out.write(bucket.name + '<br />') if 'LIST_CONTENTS' in FLAGS: srow("<h2>Listing Contents of %s bucket</h2>" % BUCKET_NAME) uri = boto.storage_uri(BUCKET_NAME, 'gs') objs = uri.get_bucket() for obj in objs: srow("gs://%s/%s <br />" % (BUCKET_NAME,obj.name)) _URLS = [ ('/', MainHandler), ('/download',DownloadHandler), ] def main(argv): application = webapp.WSGIApplication(_URLS, debug = True) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main(sys.argv)