feat: Add new gcloud commands, API clients, and third-party libraries across various services.

This commit is contained in:
2026-01-01 20:26:35 +01:00
parent 5e23cbece0
commit a19e592eb7
25221 changed files with 8324611 additions and 0 deletions

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,7 @@
Custom Runtime Definition
=========================
This is the runtime definition for GAE "custom" runtimes. Its only job is to
generate an app.yaml when it sees a Dockerfile present.

View File

@@ -0,0 +1,2 @@
env: flex
runtime: custom

View File

@@ -0,0 +1,32 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
def main(args):
if len(args) != 2:
sys.stderr.write('Invalid Usage: %s <source-root-directory>' %
sys.argv[0])
return 1
path = args[1]
if os.path.exists(os.path.join(path, 'Dockerfile')):
return 0
else:
return 1
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,11 @@
author: mmuller@google.com
name: custom
description: \
Official Google custom runtime definition.
This definition matches runtimes containing a dockerfile.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
files_to_copy:
- app.yaml

View File

@@ -0,0 +1,31 @@
#!/usr/bin/python
import os
import unittest
from gae_ext_runtime import testutil
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
class RuntimeTest(testutil.TestBase):
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTest, self).setUp()
def test_custom_runtime(self):
self.write_file('Dockerfile', 'boring contents')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
'env: flex\nruntime: custom\n')
def test_custom_runtime_no_write(self):
"""Ensure custom runtime writes app.yaml to disk with GenerateConfigData."""
self.write_file('Dockerfile', 'boring contents')
self.generate_config_data()
self.assert_file_exists_with_contents(
'app.yaml',
'env: flex\nruntime: custom\n')
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,6 @@
Go Externalized Runtime
=======================
This is the Go runtime definition that will soon be used by gcloud for
generating dockerfiles for GAE.

View File

@@ -0,0 +1,103 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Fingerprinting code for the Go runtime."""
import fnmatch
import os
import re
import sys
# TODO(b/28231839): Import a vendored gae_ext_runtime library until we have a
# better solution.
sys.path.append(os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0]))), 'lib'))
from gae_ext_runtime import comm
def find_go_files(path):
"""Return list of '*.go' files in directory 'path', listed non-recursively.
Args:
path: (str) Application path.
Returns:
([str, ...]) List of full pathnames for all '*.go' files under 'path'
dir.
"""
go_files = []
root, _, filenames = next(os.walk(path))
for filename in fnmatch.filter(filenames, '*.go'):
go_files.append(os.path.join(root, filename))
return go_files
def has_main(filename):
"""Check filename for 'package main' and 'func main'.
Args:
filename: (str) File name to check.
Returns:
(bool) True if main is found in filename.
"""
with open(filename) as f:
found_package = False
found_func = False
for line in f:
if re.match('^package main', line):
found_package = True
elif re.match('^func main', line):
found_func = True
if found_package and found_func:
return True
return False
def main(args):
if len(args) != 2:
# If we're being called incorrectly, this probably isn't happening
# from a framework.
sys.stderr.write('Invalid Usage: %s <source-root-directory>' %
sys.argv[0])
return 1
path = args[1]
comm.info('Checking for Go.')
# Test #1 - are there any '*.go' files at 'path'?
go_files = find_go_files(path)
if not go_files:
return 1
# Test #2 - check that one of these files has "package main" and "func main".
main_found = False
for f in go_files:
if has_main(f):
comm.info('Found Go main in %s', f)
main_found = True
break
if not main_found:
return 1
# Send the app.yaml contents.
runtime = 'custom' if comm.get_config().params.custom else 'go'
comm.send_runtime_params(None, {'runtime': 'go'})
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,38 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
# TODO(b/28231839): Import a vendored gae_ext_runtime library until we have a
# better solution.
sys.path.append(os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0]))), 'lib'))
from gae_ext_runtime import comm
ROOT = comm.RuntimeDefinitionRoot(os.path.dirname(os.path.dirname(__file__)))
def main(args):
params = comm.get_config().params
if params.custom or params.deploy:
comm.gen_file('Dockerfile', ROOT.read_file('data', 'Dockerfile'))
comm.gen_file('.dockerignore', ROOT.read_file('data', 'dockerignore'))
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,6 @@
# Dockerfile extending the generic Go image with application files for a
# single application.
FROM gcr.io/google-appengine/golang
COPY . /go/src/app
RUN go-wrapper install -tags appenginevm

View File

@@ -0,0 +1,5 @@
.dockerignore
Dockerfile
.git
.hg
.svn

View File

@@ -0,0 +1,14 @@
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@@ -0,0 +1,181 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Note: this file is part of the sdk-ext-runtime package. It gets copied into
# individual GAE runtime modules so that they can be easily deployed.
import json
import os
import sys
class JSONObject(object):
"""Wrapper for a JSON object.
Presents a JSON object as a python object (where fields are attributes)
instead of a dictionary. Undefined attributes produce a value of None
instead of an AttributeError.
Note that attribute names beginning with an underscore are excluded.
"""
def __getattr__(self, attr):
return None
def to_dict(self):
result = {}
for attr, val in self.__dict__.iteritems():
if not attr.startswith('_'):
result[attr] = _make_serializable(val)
return result
# Alias old style naming so this interoperates with gcloud's appinfo.
ToDict = to_dict
def _make_serializable(obj):
"""Converts objects to serializable form."""
if isinstance(obj, JSONObject):
return obj.to_dict()
else:
return obj
def _write_msg(**message):
"""Write a message to standard output.
Args:
**message: ({str: object, ...}) A JSON message encoded in keyword
arguments.
"""
json.dump(message, sys.stdout, default=_make_serializable)
sys.stdout.write('\n')
sys.stdout.flush()
def error(message, *args):
_write_msg(type='error', message=message % args)
def warn(message, *args):
_write_msg(type='warn', message=message % args)
def info(message, *args):
_write_msg(type='info', message=message % args)
def debug(message, *args):
_write_msg(type='debug', message=message % args)
def print_status(message, *args):
_write_msg(type='print_status', message=message % args)
def send_runtime_params(params, appinfo=None):
"""Send runtime parameters back to the controller.
Args:
params: ({str: object, ...}) Set of runtime parameters. Must be
json-encodable.
appinfo: ({str: object, ...} or None) Contents of the app.yaml file to
be produced by the runtime definition. Required fields may be
added to this by the framework, the only thing an application
needs to provide is the "runtime" field and any additional data
fields.
"""
if appinfo is not None:
_write_msg(type='runtime_parameters', runtime_data=params,
appinfo=appinfo)
else:
_write_msg(type='runtime_parameters', runtime_data=params)
def set_docker_context(path):
"""Send updated Docker context to the controller.
Args:
path: (str) new directory to use as docker context.
"""
_write_msg(type='set_docker_context', path=path)
def get_config():
"""Request runtime parameters from the controller.
Returns:
(object) The runtime parameters represented as an object.
"""
_write_msg(type='get_config')
return dict_to_object(json.loads(sys.stdin.readline()))
def dict_to_object(json_dict):
"""Converts a dictionary to a python object.
Converts key-values to attribute-values.
Args:
json_dict: ({str: object, ...})
Returns:
(JSONObject)
"""
obj = JSONObject()
for name, val in json_dict.iteritems():
if isinstance(val, dict):
val = dict_to_object(val)
setattr(obj, name, val)
return obj
class RuntimeDefinitionRoot(object):
"""Abstraction that allows us to access files in the runtime definiton."""
def __init__(self, path):
self.root = path
def read_file(self, *name):
with open(os.path.join(self.root, *name)) as src:
return src.read()
def gen_file(name, contents):
"""Generate the file.
This writes the file to be generated back to the controller.
Args:
name: (str) The UNIX-style relative path of the file.
contents: (str) The complete file contents.
"""
_write_msg(type='gen_file', filename=name, contents=contents)
def query_user(prompt, default=None):
"""Query the user for data.
Args:
prompt: (str) Prompt to display to the user.
default: (str or None) Default value to use if the user doesn't input
anything.
Returns:
(str) Value returned by the user.
"""
kwargs = {}
kwargs['prompt'] = prompt
if default is not None:
kwargs['default'] = default
_write_msg(type='query_user', **kwargs)
return json.loads(sys.stdin.readline())['result']

View File

@@ -0,0 +1,8 @@
author: mmuller@google.com
name: go
description: Official Google Go runtime definition.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,133 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import unittest
import yaml
from gae_ext_runtime import testutil
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
class RuntimeTests(testutil.TestBase):
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTests, self).setUp()
def assert_no_file(self, filename):
"""Asserts that the relative path 'filename' does not exist."""
self.assertFalse(os.path.exists(os.path.join(self.temp_path, filename)))
def test_go_files_no_go(self):
self.write_file('foo.notgo', 'package main\nfunc main')
self.assertFalse(self.generate_configs())
self.assertEqual(os.listdir(self.temp_path), ['foo.notgo'])
def test_go_files_with_go(self):
self.write_file('foo.go', 'package main\nfunc main')
self.generate_configs()
with open(self.full_path('app.yaml')) as f:
contents = yaml.load(f)
self.assertEqual(contents,
{'runtime': 'go', 'env': 'flex'})
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
self.generate_configs(deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.read_runtime_def_file('data', 'Dockerfile'))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_go_files_in_subdirs(self):
"""Test go runtime does not recognize go files in subdirectories."""
subdir_path = os.mkdir(os.path.join(self.temp_path, 'subdir'))
self.write_file(os.path.join('subdir', 'foo.go'),
'package main\nfunc main')
self.assertEqual(None, self.generate_configs())
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
def test_go_genfiles_with_go(self):
"""Test generate_config_data with single .go file."""
self.write_file('foo.go', 'package main\nfunc main')
self.generate_configs()
with open(self.full_path('app.yaml')) as f:
contents = yaml.load(f)
self.assertEqual(contents,
{'runtime': 'go', 'env': 'flex'})
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
cfg_files = self.generate_config_data(deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.read_runtime_def_file('data', 'Dockerfile'))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_go_custom_runtime(self):
self.write_file('foo.go', 'package main\nfunc main')
self.generate_configs(custom=True)
self.assert_file_exists_with_contents(
'app.yaml',
'env: flex\nruntime: go\n')
self.assert_file_exists_with_contents(
'Dockerfile',
self.read_runtime_def_file('data', 'Dockerfile'))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_go_custom_runtime_no_write(self):
"""Test generate_config_data with custom runtime."""
self.write_file('foo.go', 'package main\nfunc main')
cfg_files = self.generate_config_data(custom=True)
self.assert_file_exists_with_contents(
'app.yaml',
'env: flex\nruntime: go\n')
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.read_runtime_def_file('data', 'Dockerfile'))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_go_runtime_field(self):
self.write_file('foo.go', 'package main\nfunc main')
config = testutil.AppInfoFake(
runtime="go",
env=2)
self.assertTrue(self.generate_configs(appinfo=config,
deploy=True))
def test_go_custom_runtime_field(self):
self.write_file('foo.go', 'package main\nfunc main')
config = testutil.AppInfoFake(
runtime="custom",
env=2)
self.assertTrue(self.generate_configs(appinfo=config,
deploy=True))
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,98 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Java detection plugin."""
import os
import sys
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
def main(args):
if len(args) != 2:
# If we're being called incorrectly, this probably isn't happening
# from a framework.
sys.stderr.write('Invalid Usage: %s <source-root-directory>' % sys.argv[0])
return 1
# Get the first argument, should be the source root directory.
path = args[1]
entrypoint = None
server = None
openjdk = None
config = comm.get_config()
appinfo = config.params.appinfo
if appinfo and appinfo.entrypoint:
entrypoint = appinfo.entrypoint
comm.info('Checking for Java.')
if appinfo:
runtime_config = appinfo.runtime_config
if runtime_config:
for key, value in runtime_config.to_dict().iteritems():
if key == 'server':
if value != 'jetty9':
comm.error('Unknown server : %s.' % value)
return 1
server = value
elif key == 'jdk':
if value != 'openjdk8':
comm.error('Unknown JDK : %s.' % value)
return 1
openjdk = value
else:
comm.error('Unknown runtime_config entry : %s.' % key)
return 1
artifact_to_deploy = '?'
# check for any Java known artifacts: a jar, a war, or an exploded Web App.
# TODO(ludo): expand to more complex configs with multiple Jars.
number_of_possible_artifacts = 0
for filename in os.listdir(path):
if filename.endswith('.war'):
artifact_to_deploy = filename
number_of_possible_artifacts += 1
if filename.endswith('.jar'):
artifact_to_deploy = filename
number_of_possible_artifacts += 1
if filename.endswith('WEB-INF'):
artifact_to_deploy = '.'
number_of_possible_artifacts += 1
if number_of_possible_artifacts == 0:
return 1
if number_of_possible_artifacts > 1:
comm.error('Too many java artifacts to deploy (.jar, .war, or Java '
'Web App).')
return 1
if not appinfo:
appinfo = {'runtime': 'custom' if config.params.custom else 'java'}
comm.send_runtime_params({'entrypoint': entrypoint, 'server': server,
'openjdk': openjdk,
'artifact_to_deploy': artifact_to_deploy},
appinfo)
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,141 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Java config generation plugin."""
import os
import StringIO
import sys
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
import constants
ROOT = comm.RuntimeDefinitionRoot(ROOT_DIR)
def is_flex(env):
return env in ['2', 'flex', 'flexible']
class JavaConfigurator(object):
"""Generates configuration for a Java application.
What is supported is:
- jar file (run with Open JDK8 image)
- war file (run with Jetty9 image)
- Exploded war directory (with WEB-INF/):
- if env: 2, we use the latest Jetty9 compat runtime image
- if not, we use the current Jetty9 compat image we build.
This will ease the transition to the new Jetty9 compat runtime for
people migrating to env: 2. Once all are on env: 2, we will remove
entirely the support for the legacy Jetty9 compat runtime.
"""
def __init__(self, appinfo, deploy, entrypoint, server,
openjdk, artifact_to_deploy, custom):
"""Constructor.
Args:
appinfo: (apphosting.api.appinfo.AppInfoExternal or None) The parsed
app.yaml file for the module if it exists.
deploy: (bool) True if run in deployment mode.
entrypoint: (str) Name of the entrypoint to generate.
server: (str) Name of the server to use (jetty9 or None for now).
openjdk: (str) Name of the jdk to use (openjdk8 or None for now).
artifact_to_deploy: (str) Name of the file or directory to deploy.
custom: (bool) True if it is a custom runtime.
"""
self.appinfo = appinfo
self.deploy = deploy
self.custom = custom
self.entrypoint = entrypoint
self.server = server
self.openjdk = openjdk
self.artifact_to_deploy = artifact_to_deploy
def GenerateConfigs(self):
"""Generates config files for the module, not including app.yaml.
For app.yaml generation, see gae-xrt-java/bin/detect.
"""
if self.custom or self.deploy:
self._GenerateDockerfile()
self._GenerateDockerignore()
def _GenerateDockerfile(self):
"""Generates a Dockerfile appropriate to this application.
Raises:
JavaConfigError: if there is an app.yaml configuration error.
"""
# Customize the dockerfile.
out = StringIO.StringIO()
if self.artifact_to_deploy.endswith('.war'):
if self.appinfo and is_flex(self.appinfo.env):
out.write(constants.DOCKERFILE_JETTY_PREAMBLE)
else:
out.write(constants.DOCKERFILE_JETTY9_PREAMBLE)
out.write(constants.DOCKERFILE_INSTALL_WAR.format(
self.artifact_to_deploy))
if self.artifact_to_deploy.endswith('.jar'):
if self.server is not None:
raise JavaConfigError('Cannot use server %s '
'for jar deployment.' % self.server)
if self.appinfo and is_flex(self.appinfo.env):
out.write(constants.DOCKERFILE_JAVA_PREAMBLE)
else:
out.write(constants.DOCKERFILE_JAVA8_PREAMBLE)
out.write(constants.DOCKERFILE_INSTALL_APP.format(
self.artifact_to_deploy))
if self.artifact_to_deploy == '.':
if self.appinfo and is_flex(self.appinfo.env):
out.write(constants.DOCKERFILE_COMPAT_PREAMBLE)
elif self.openjdk == 'openjdk8':
out.write(constants.DOCKERFILE_COMPAT_PREAMBLE)
else:
out.write(constants.DOCKERFILE_LEGACY_PREAMBLE)
out.write(constants.DOCKERFILE_INSTALL_APP.format(
self.artifact_to_deploy))
# Generate the appropriate start command.
if self.entrypoint:
out.write(constants.DOCKERFILE_CMD % self.entrypoint)
elif self.artifact_to_deploy.endswith('.jar'):
# for jar execution generate the command to run:
out.write(constants.DOCKERFILE_JAVA8_JAR_CMD.format(
self.artifact_to_deploy))
comm.gen_file('Dockerfile', out.getvalue())
def _GenerateDockerignore(self):
"""Generates a .dockerignore file appropriate to this application."""
comm.gen_file('.dockerignore', ROOT.read_file('data', 'dockerignore'))
def main(args):
config = comm.get_config()
JavaConfigurator(config.params.appinfo,
config.params.deploy,
config.runtime_data.entrypoint,
config.runtime_data.server,
config.runtime_data.openjdk,
config.runtime_data.artifact_to_deploy,
config.params.custom).GenerateConfigs()
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,6 @@
.dockerignore
Dockerfile
.git
.hg
.svn
app.yaml

View File

@@ -0,0 +1,174 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Note: this file is part of the sdk-ext-runtime package. It gets copied into
# individual GAE runtime modules so that they can be easily deployed.
import json
import os
import sys
class JSONObject(object):
"""Wrapper for a JSON object.
Presents a JSON object as a python object (where fields are attributes)
instead of a dictionary. Undefined attributes produce a value of None
instead of an AttributeError.
Note that attribute names beginning with an underscore are excluded.
"""
def __getattr__(self, attr):
return None
def to_dict(self):
result = {}
for attr, val in self.__dict__.iteritems():
if not attr.startswith('_'):
result[attr] = _make_serializable(val)
return result
# Alias old style naming so this interoperates with gcloud's appinfo.
ToDict = to_dict
def _make_serializable(obj):
"""Converts objects to serializable form."""
if isinstance(obj, JSONObject):
return obj.to_dict()
else:
return obj
def _write_msg(**message):
"""Write a message to standard output.
Args:
**message: ({str: object, ...}) A JSON message encoded in keyword
arguments.
"""
json.dump(message, sys.stdout, default=_make_serializable)
sys.stdout.write('\n')
sys.stdout.flush()
def error(message, *args):
_write_msg(type='error', message=message % args)
def warn(message, *args):
_write_msg(type='warn', message=message % args)
def info(message, *args):
_write_msg(type='info', message=message % args)
def debug(message, *args):
_write_msg(type='debug', message=message % args)
def print_status(message, *args):
_write_msg(type='print_status', message=message % args)
def send_runtime_params(params, appinfo=None):
"""Send runtime parameters back to the controller.
Args:
params: ({str: object, ...}) Set of runtime parameters. Must be
json-encodable.
appinfo: ({str: object, ...} or None) Contents of the app.yaml file to
be produced by the runtime definition. Required fields may be
added to this by the framework, the only thing an application
needs to provide is the "runtime" field and any additional data
fields.
"""
if appinfo is not None:
_write_msg(type='runtime_parameters', runtime_data=params,
appinfo=appinfo)
else:
_write_msg(type='runtime_parameters', runtime_data=params)
def get_config():
"""Request runtime parameters from the controller.
Returns:
(object) The runtime parameters represented as an object.
"""
_write_msg(type='get_config')
return dict_to_object(json.loads(sys.stdin.readline()))
def dict_to_object(json_dict):
"""Converts a dictionary to a python object.
Converts key-values to attribute-values.
Args:
json_dict: ({str: object, ...})
Returns:
(JSONObject)
"""
obj = JSONObject()
for name, val in json_dict.iteritems():
if isinstance(val, dict):
val = dict_to_object(val)
setattr(obj, name, val)
return obj
class RuntimeDefinitionRoot(object):
"""Abstraction that allows us to access files in the runtime definiton."""
def __init__(self, path):
self.root = path
def read_file(self, *name):
with open(os.path.join(self.root, *name)) as src:
return src.read()
def gen_file(name, contents):
"""Generate the file.
This writes the file to be generated back to the controller.
Args:
name: (str) The UNIX-style relative path of the file.
contents: (str) The complete file contents.
"""
_write_msg(type='gen_file', filename=name, contents=contents)
def query_user(prompt, default=None):
"""Query the user for data.
Args:
prompt: (str) Prompt to display to the user.
default: (str or None) Default value to use if the user doesn't input
anything.
Returns:
(str) Value returned by the user.
"""
kwargs = {}
kwargs['prompt'] = prompt
if default is not None:
kwargs['default'] = default
_write_msg(type='query_user', **kwargs)
return json.loads(sys.stdin.readline())['result']

View File

@@ -0,0 +1,30 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Constants shared by plugin and tests."""
# rutimes based on appengine-java-vm-runtime GitHub repo
DOCKERFILE_JAVA8_PREAMBLE = 'FROM gcr.io/google_appengine/openjdk8\n'
DOCKERFILE_JETTY9_PREAMBLE = 'FROM gcr.io/google_appengine/jetty9\n'
# runtimes based on openjdk-runtime and jetty-runtime GitHub repos
DOCKERFILE_JAVA_PREAMBLE = 'FROM gcr.io/google_appengine/openjdk\n'
DOCKERFILE_JETTY_PREAMBLE = 'FROM gcr.io/google_appengine/jetty\n'
DOCKERFILE_LEGACY_PREAMBLE = 'FROM gcr.io/google_appengine/java-compat\n'
DOCKERFILE_COMPAT_PREAMBLE = 'FROM gcr.io/google_appengine/jetty9-compat\n'
DOCKERFILE_CMD = 'CMD {0}\n'
DOCKERFILE_JAVA8_JAR_CMD = 'CMD ["java", "-jar", "/app/{0}"]\n'
DOCKERFILE_INSTALL_APP = 'ADD {0} /app/\n'
DOCKERFILE_INSTALL_WAR = 'ADD {0} $JETTY_BASE/webapps/root.war\n'

View File

@@ -0,0 +1,22 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: java
author: mmuller@google.com
description: Official Google Java runtime definition.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,555 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests of the java runtime."""
import logging
import mock
import os
import re
import sys
import shutil
import tempfile
import textwrap
import unittest
from gae_ext_runtime import testutil
from gae_ext_runtime import ext_runtime
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import constants
#from googlecloudsdk.third_party.appengine.api import appinfo
#from googlecloudsdk.core import log
#from googlecloudsdk.api_lib.app import ext_runtime
#from googlecloudsdk.api_lib.app.runtimes import fingerprinter
#from googlecloudsdk.api_lib.app.ext_runtimes import fingerprinting
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
class RuntimeTests(testutil.TestBase):
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTests, self).setUp()
def assert_no_file(self, *path):
self.assertFalse(os.path.exists(self.full_path(*path)))
def make_app_yaml(self, runtime):
return 'env: flex\nruntime: {runtime}\n'.format(runtime=runtime)
def test_java_all_defaults(self):
self.write_file('foo.jar', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
self.generate_configs(deploy=True)
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assertEqual(set(os.listdir(self.temp_path)),
{'.dockerignore', 'Dockerfile', 'app.yaml', 'foo.jar'})
def test_java_all_defaults_no_write(self):
"""Test generate_config_data after writing app.yaml."""
self.write_file('foo.jar', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
cfg_files = self.generate_config_data(deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assertEqual(set(os.listdir(self.temp_path)),
{'app.yaml', 'foo.jar'})
def test_java_custom(self):
self.write_file('foo.jar', '')
self.generate_configs(deploy=False, custom=True)
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('custom'))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
def test_java_custom_no_write(self):
"""Test generate_config_data with custom=True.
app.yaml should be written to disk. Also tests correct dockerfile
contents with a .jar.
"""
self.write_file('foo.jar', '')
cfg_files = self.generate_config_data(deploy=False, custom=True)
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('custom'))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
def test_java_files_no_java(self):
self.write_file('foo.nojava', '')
self.assertFalse(self.generate_configs())
self.assertEqual(os.listdir(self.temp_path), ['foo.nojava'])
def test_java_files_with_war(self):
self.write_file('foo.war', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
self.generate_configs(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JETTY_PREAMBLE,
constants.DOCKERFILE_INSTALL_WAR.format('foo.war'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore', self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_war_no_write(self):
"""Test generate_config_data Dockerfile contents with .war file."""
self.write_file('foo.war', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
cfg_files = self.generate_config_data(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JETTY_PREAMBLE,
constants.DOCKERFILE_INSTALL_WAR.format('foo.war'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_jar(self):
self.write_file('foo.jar', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
self.generate_configs(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_jar_no_write(self):
"""Test generate_config_data Dockerfile contents with .jar file."""
self.write_file('foo.jar', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
cfg_files = self.generate_config_data(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JAVA_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('foo.jar'),
constants.DOCKERFILE_JAVA8_JAR_CMD.format('foo.jar'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_webinf(self):
self.write_file('WEB-INF', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
self.generate_configs(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_webinf_no_write(self):
"""Test generate_config_data Dockerfile contents with 'WEB-INF' file."""
self.write_file('WEB-INF', '')
self.generate_configs()
self.assert_file_exists_with_contents('app.yaml',
self.make_app_yaml('java'))
self.assert_no_file('Dockerfile')
self.assert_no_file('.dockerignore')
cfg_files = self.generate_config_data(deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_too_many_artifacts(self):
self.write_file('WEB-INF', '')
self.write_file('foo.jar', '')
errors = []
def ErrorFake(message):
errors.append(message)
with mock.patch.dict(ext_runtime._LOG_FUNCS, {'error': ErrorFake}):
self.assertFalse(self.generate_configs())
self.assertEqual(errors,
['Too many java artifacts to deploy (.jar, .war, or '
'Java Web App).'])
# TODO(b/63385240): cover the case were the app.yaml might not be called
# app.yaml. For example: "gen-config --config=foo.yaml <path>"
# will use foo.yaml
def test_java_files_with_war_and_yaml(self):
self.write_file('foo.war', '')
appinfo = testutil.AppInfoFake(
runtime='java',
env='flex',
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
self.generate_configs(appinfo=appinfo, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JETTY_PREAMBLE,
constants.DOCKERFILE_INSTALL_WAR.format('foo.war'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_war_and_yaml_no_write(self):
"""Test generate_config_data with .war and fake appinfo."""
self.write_file('foo.war', '')
appinfo = testutil.AppInfoFake(
runtime='java',
env='2',
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
cfg_files = self.generate_config_data(appinfo=appinfo, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_JETTY_PREAMBLE,
constants.DOCKERFILE_INSTALL_WAR.format('foo.war'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_env2(self):
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
env='2',
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
self.generate_configs(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_env2_no_write(self):
"""Test generate_config_data with .war, fake appinfo, env=2."""
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
env='2',
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_no_env2(self):
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
vm=True,
runtime_config=dict(server='jetty9'))
self.generate_configs(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_LEGACY_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_no_env2_no_write(self):
"""Test generate_config_data with .war, fake appinfo, env != 2."""
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
vm=True,
runtime_config=dict(server='jetty9'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_LEGACY_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_open_jdk8_no_env2(self):
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
vm=True,
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
self.generate_configs(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_web_inf_and_yaml_and_open_jdk8_no_env2_no_write(
self):
"""Test generate_config_data with WEB-INF file, fake appinfo."""
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java',
vm=True,
runtime_config=dict(
jdk='openjdk8',
server='jetty9'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_COMPAT_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java_files_with_config_error(self):
self.write_file('foo.war', '')
errors = []
def ErrorFake(message):
errors.append(message)
config = testutil.AppInfoFake(
runtime='java',
env='2',
runtime_config=dict(
jdk='openjdk9'))
with mock.patch.dict(ext_runtime._LOG_FUNCS, {'error': ErrorFake}):
self.assertFalse(self.generate_configs(appinfo=config, deploy=True))
self.assertEqual(errors, ['Unknown JDK : openjdk9.'])
def test_java_custom_runtime_field(self):
self.write_file('foo.jar', '')
config = testutil.AppInfoFake(
runtime='java',
env='2')
self.assertTrue(self.generate_configs(appinfo=config, deploy=True))
def test_java7_runtime(self):
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java7',
vm=True)
self.generate_configs(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_LEGACY_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_file_exists_with_contents('Dockerfile',
''.join(dockerfile_contents))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_java7_runtime_no_write(self):
"""Test generate_config_data with java7 runtime."""
self.write_file('WEB-INF', '')
config = testutil.AppInfoFake(
runtime='java7',
vm=True)
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
dockerfile_contents = [
constants.DOCKERFILE_LEGACY_PREAMBLE,
constants.DOCKERFILE_INSTALL_APP.format('.'),
]
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
''.join(dockerfile_contents))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_runtime_def_file('data', 'dockerignore'))
def test_detect_appinfo_war(self):
self.write_file('foo.war', '')
configurator = self.detect()
self.assertEqual(configurator.generated_appinfo, {'runtime':'java',
'env': 'flex'})
def test_detect_appinfo_jar(self):
self.write_file('foo.jar', '')
configurator = self.detect()
self.assertEqual(configurator.generated_appinfo, {'runtime':'java',
'env': 'flex'})
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,24 @@
Want to contribute? Great! First, read this page (including the small print at the end).
### Before you contribute
Before we can use your code, you must sign the
[Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1)
(CLA), which you can do online. The CLA is necessary mainly because you own the
copyright to your changes, even after your contribution becomes part of our
codebase, so we need your permission to use and distribute your code. We also
need to be sure of various other things—for instance that you'll tell us if you
know that your code infringes on other people's patents. You don't have to sign
the CLA until after you've submitted your code for review and a member has
approved it, but you must do it before we can put your code into our codebase.
Before you start working on a larger contribution, you should get in touch with
us first through the issue tracker with your idea so that we can help out and
possibly guide you. Coordinating up front makes it much easier to avoid
frustration later on.
### Code reviews
All submissions, including submissions by project members, require review. We
use Github pull requests for this purpose.
### The small print
Contributions made by corporations are covered by a different agreement than
the one above, the Software Grant and Corporate Contributor License Agreement.

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,6 @@
Node.js Externalized Runtime
============================
This is the node.js runtime definition that will soon be used by gcloud for
generating dockerfiles for GAE.

View File

@@ -0,0 +1,145 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Language detection script.
import json
import os
import re
import sys
import subprocess
DEV_NULL = open(os.devnull, 'w')
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
def asList(val):
"""Returns the value if it is a list else returns a list with the value.
"""
return val if isinstance(val, list) else [val]
def main(args):
if len(args) != 2:
# If we're being called incorrectly, this probably isn't happening from a
# framework.
sys.stderr.write('Invalid Usage: %s <source-root-directory>' %
sys.argv[0])
return 1
# Get the first argument, should be the source root directory.
path = args[1]
# Show failures in debug or error depending on whether nodejs has been
# explicitly specified.
config = comm.get_config()
if ((config.params.appinfo and config.params.appinfo.runtime) or
config.params.runtime):
log_detect_error = comm.error
else:
log_detect_error = comm.debug
comm.info('Checking for Node.js.')
package_json = os.path.join(path, 'package.json')
yarn_lock_name = 'yarn.lock'
yarn_lock = os.path.join(path, yarn_lock_name)
got_shrinkwrap = False
if not os.path.isfile(package_json):
comm.debug('node.js checker: No package.json file.')
got_package_json = False
got_scripts_start = False
node_version = None
use_yarn = False
else:
got_package_json = True
# If there's a package.json file, that's unusual enough that any
# errors should be surfaced to the user.
log_detect_error = comm.error
# Consider the yarn.lock file as present if and only if the yarn.lock
# file exists and is not specified as being skipped in app.yaml.
try:
raw_skip_files = config.params.appinfo.skip_files
skip_files = [] if raw_skip_files is None else asList(raw_skip_files)
except AttributeError:
# If the skip_file attribute or any other parent attribute
# does not exist, treat the skip_files as not user specified.
skip_files = []
yarn_lock_exists = os.path.isfile(yarn_lock)
yarn_lock_skipped = False
for pattern in skip_files:
if re.match(pattern, yarn_lock_name) is not None:
yarn_lock_skipped = True
break;
use_yarn = yarn_lock_exists and not yarn_lock_skipped
# Try to read the package.json file.
try:
with open(package_json) as f:
contents = json.load(f)
except (IOError, ValueError) as ex:
# If we have an invalid or unreadable package.json file, there's
# something funny going on here so fail recognition.
# A package.json that exists is unusual enough that we want to warn
# regardless of whether the nodejs runtime was specified.
log_detect_error('node.js checker: error accessing package.json: '
'%r' % ex)
return 1
# See if we've got a scripts.start field.
got_scripts_start = bool(contents.get('scripts', {}).get('start'))
# See if a version of node is specified.
try:
node_version = contents.get('engines', {}).get('node', None)
comm.info('node version is %s', node_version)
except AttributeError:
# Most likely "engines" wasn't a dictionary.
comm.warn('node.js checker: ignoring invalid "engines" field in '
'package.json')
node_version = None
if node_version is None:
comm.warn('No node version specified. Please add your node '
'version, see '
'https://docs.npmjs.com/files/package.json#engines')
if got_scripts_start or os.path.exists(os.path.join(path, 'server.js')):
runtime = 'custom' if config.params.custom else 'nodejs'
appinfo = {'runtime': runtime,
'env': 'flex'}
comm.send_runtime_params({'got_package_json': got_package_json,
'got_scripts_start': got_scripts_start,
'node_version': node_version,
'use_yarn': use_yarn},
appinfo=appinfo)
else:
log_detect_error('node.js checker: Neither "start" in the "scripts" '
'section of "package.json" nor the "server.js" file '
'were found.')
return 1
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,90 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import StringIO
import pipes
import sys
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
ROOT = comm.RuntimeDefinitionRoot(ROOT_DIR)
# The template for the docker command that is injected into the Dockerfile
# when the user specifies a node version in their "engines" section.
INSTALL_NODE_TEMPLATE = ROOT.read_file('data/install-node-version')
def main(args):
"""Generate all config files for the module."""
# Get runtime parameters from the controller.
info = comm.get_config()
runtime_data = info.runtime_data
params = info.params
# Generate app.yaml.
if not params.appinfo:
app_yaml = 'app.yaml'
runtime = 'custom' if params.custom else 'nodejs'
app_yaml_contents = ROOT.read_file('data/app.yaml')
comm.gen_file(app_yaml,
app_yaml_contents.format(runtime=runtime))
if params.custom or params.deploy:
dockerfile = 'Dockerfile'
# Customize the dockerfile.
out = StringIO.StringIO()
out.write(ROOT.read_file('data', 'Dockerfile'))
if runtime_data.node_version:
# Let node check to see if it satisfies the version constraint and
# try to install the correct version if not.
out.write(INSTALL_NODE_TEMPLATE %
{'version_spec': pipes.quote(runtime_data.node_version)})
# If the directory structure indicates that yarn is being used
# then install yarn since (unlike npm) Node.js doesn't include it
if runtime_data.use_yarn:
out.write(ROOT.read_file('data', 'install-yarn'))
out.write('COPY . /app/\n')
# Generate npm or yarn install if there is a package.json.
if runtime_data.got_package_json:
out.write(
ROOT.read_file('data/{0}-package-json-install'.
format('yarn' if runtime_data.use_yarn
else 'npm')))
# Generate the appropriate start command.
if runtime_data.got_scripts_start:
out.write(
'CMD {0} start\n'.format('yarn' if runtime_data.use_yarn
else 'npm'))
else:
out.write('CMD node server.js\n')
comm.gen_file('Dockerfile', out.getvalue())
# Generate .dockerignore
dockerignore = '.dockerignore'
comm.gen_file(dockerignore, ROOT.read_file('data/dockerignore'))
if __name__ == '__main__':
main(sys.argv)

View File

@@ -0,0 +1,3 @@
# Dockerfile extending the generic Node image with application files for a
# single application.
FROM gcr.io/google_appengine/nodejs

View File

@@ -0,0 +1,2 @@
env: flex
runtime: {runtime}

View File

@@ -0,0 +1,22 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
node_modules
.dockerignore
Dockerfile
npm-debug.log
yarn-error.log
.git
.hg
.svn

View File

@@ -0,0 +1,4 @@
# Check to see if the the version included in the base runtime satisfies
# %(version_spec)s, if not then do an npm install of the latest available
# version that satisfies it.
RUN /usr/local/bin/install_node %(version_spec)s

View File

@@ -0,0 +1,6 @@
# You have to specify "--unsafe-perm" with npm install
# when running as root. Failing to do this can cause
# install to appear to succeed even if a preinstall
# script fails, and may have other adverse consequences
# as well.
RUN npm install --unsafe-perm --global yarn

View File

@@ -0,0 +1,11 @@
# You have to specify "--unsafe-perm" with npm install
# when running as root. Failing to do this can cause
# install to appear to succeed even if a preinstall
# script fails, and may have other adverse consequences
# as well.
# This command will also cat the npm-debug.log file after the
# build, if it exists.
RUN npm install --unsafe-perm || \
((if [ -f npm-debug.log ]; then \
cat npm-debug.log; \
fi) && false)

View File

@@ -0,0 +1,5 @@
RUN yarn install --production || \
((if [ -f yarn-error.log ]; then \
cat yarn-error.log; \
fi) && false)

View File

@@ -0,0 +1,121 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import os
import sys
def _write_msg(**message):
"""Write a message to standard output.
Args:
**message: ({str: object, ...}) A JSON message encoded in keyword
arguments.
"""
json.dump(message, sys.stdout)
sys.stdout.write('\n')
sys.stdout.flush()
def error(message, *args):
_write_msg(type='error', message=message % args)
def warn(message, *args):
_write_msg(type='warn', message=message % args)
def info(message, *args):
_write_msg(type='info', message=message % args)
def debug(message, *args):
_write_msg(type='debug', message=message % args)
def print_status(message, *args):
_write_msg(type='print_status', message=message % args)
def send_runtime_params(params, appinfo=None):
"""Send runtime parameters back to the controller.
Args:
params: ({str: object, ...}) Set of runtime parameters. Must be
json-encodable.
appinfo: ({str: object, ...}) Contents of the app.yaml file to
be produced by the runtime definition. Required fields may be
added to this by the framework, the only thing an application
needs to provide is the "runtime" field and any additional data
fields.
"""
if appinfo is not None:
_write_msg(type='runtime_parameters', runtime_data=params,
appinfo=appinfo)
else:
_write_msg(type='runtime_parameters', runtime_data=params)
def get_config():
"""Request runtime parameters from the controller.
Returns:
(object) The runtime parameters represented as an object.
"""
_write_msg(type='get_config')
return dict_to_object(json.loads(sys.stdin.readline()))
def dict_to_object(json_dict):
"""Converts a dictionary to a python object.
Converts key-values to attribute-values.
Args:
json_dict: ({str: object, ...})
Returns:
(object)
"""
class Object(object):
pass
obj = Object()
for name, val in json_dict.iteritems():
if isinstance(val, dict):
val = dict_to_object(val)
setattr(obj, name, val)
return obj
class RuntimeDefinitionRoot(object):
"""Abstraction that allows us to access files in the runtime definiton."""
def __init__(self, path):
self.root = path
def read_file(self, *name):
with open(os.path.join(self.root, *name)) as src:
return src.read()
def gen_file(name, contents):
"""Generate the file.
This writes the file to be generated back to the controller.
Args:
name: (str) The UNIX-style relative path of the file.
contents: (str) The complete file contents.
"""
_write_msg(type='gen_file', filename=name, contents=contents)

View File

@@ -0,0 +1,8 @@
author: mmuller@google.com
name: nodejs
description: Official Google Node.js runtime definition.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,480 @@
#!/usr/bin/python
import mock
import os
import re
import sys
import stat
import shutil
import tempfile
import textwrap
import unittest
from gae_ext_runtime import ext_runtime
from gae_ext_runtime import testutil
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
class RuntimeTests(testutil.TestBase):
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTests, self).setUp()
def read_dist_file(self, *args):
"""Read the entire contents of the file.
Returns the entire contents of the file identified by a set of
arguments forming a path relative to the root of the runtime
definition.
TODO: Move this down into the SDK.
Args:
*args: A set of path components (see full_path()). Note that
these are relative to the runtime definition root, not the
temporary directory.
"""
with open(os.path.join(self.runtime_def_root, *args)) as fp:
return fp.read()
def test_node_js_server_js_only(self):
self.write_file('server.js', 'fake contents')
self.generate_configs()
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='nodejs'))
self.generate_configs(deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.read_dist_file('data', 'Dockerfile') + textwrap.dedent("""\
COPY . /app/
CMD node server.js
"""))
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_dist_file('data', 'dockerignore'))
self.assertEqual(set(os.listdir(self.temp_path)),
{'Dockerfile', '.dockerignore', 'app.yaml',
'server.js'})
def test_node_js_server_js_only_no_write(self):
"""Test generate_config_data with only .js files.
After running generate_configs(), app.yaml exists; after
generate_config_data(), only app.yaml should exist on disk --
Dockerfile and .dockerignore should be returned by the method."""
self.write_file('server.js', 'fake contents')
self.generate_configs()
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='nodejs'))
cfg_files = self.generate_config_data(deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.read_dist_file('data', 'Dockerfile') + textwrap.dedent("""\
COPY . /app/
CMD node server.js
"""))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_dist_file('data', 'dockerignore'))
self.assertEqual(set(os.listdir(self.temp_path)),
{'app.yaml', 'server.js'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def _validate_docker_files_for_npm(self):
base_dockerfile = self.read_dist_file('data', 'Dockerfile')
self.assert_file_exists_with_contents(
'Dockerfile',
base_dockerfile + 'COPY . /app/\n' +
self.read_dist_file('data', 'npm-package-json-install') +
'CMD npm start\n')
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_dist_file('data', 'dockerignore'))
def test_node_js_package_json_npm(self):
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.generate_configs()
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='nodejs'))
self.generate_configs(deploy=True)
self._validate_docker_files_for_npm()
self.assertEqual(set(os.listdir(self.temp_path)),
{'Dockerfile', '.dockerignore', 'app.yaml',
'foo.js', 'package.json'})
def _validate_docker_files_for_yarn(self):
base_dockerfile = self.read_dist_file('data', 'Dockerfile')
install_yarn = self.read_dist_file('data', 'install-yarn')
self.assert_file_exists_with_contents(
'Dockerfile',
base_dockerfile + install_yarn + 'COPY . /app/\n' +
self.read_dist_file('data', 'yarn-package-json-install') +
'CMD yarn start\n')
self.assert_file_exists_with_contents(
'.dockerignore',
self.read_dist_file('data', 'dockerignore'))
def test_node_js_package_json_yarn(self):
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.write_file('yarn.lock', 'yarn overridden')
self.generate_configs()
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='nodejs'))
self.generate_configs(deploy=True)
self._validate_docker_files_for_yarn()
self.assertEqual(set(os.listdir(self.temp_path)),
{'Dockerfile', '.dockerignore', 'app.yaml',
'foo.js', 'package.json', 'yarn.lock'})
def _validate_file_list_for_skip_yarn_lock(self):
self.assertEqual(set(os.listdir(self.temp_path)),
{'Dockerfile', '.dockerignore', 'yarn.lock',
'foo.js', 'package.json'})
def test_skip_yarn_lock_with_other_files(self):
"""Ensure use_yarn is False with yarn.lock present but is being skipped.
Further, this test verifies that use_yarn is False even if multiple
other entries are present in skip_files.
A yarn executable is injected that passes all checks to ensure that if
yarn.lock is set to be skipped, use_yarn is set to False even if yarn
can be executed and reports that the yarn.lock file is valid.
"""
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.write_file('foo.js', 'fake contents')
self.write_file('yarn.lock', 'fake contents')
config = testutil.AppInfoFake(runtime='nodejs',
skip_files=['^abc$',
'^xyz$',
'^yarn\.lock$',
'^node_modules$'])
configurator = self.detect(appinfo=config)
self.assertEqual(configurator.data['use_yarn'], False)
self.generate_configs(appinfo=config, deploy=True)
self._validate_docker_files_for_npm()
self._validate_file_list_for_skip_yarn_lock()
def test_only_skip_yarn_lock(self):
"""Ensure use_yarn is False with yarn.lock present but is being skipped.
Further, this test ensures use_yarn is false if the value obtained
from skip_files is a regex string and not a list of strings.
A yarn executable is injected that passes all checks to ensure that if
yarn.lock is set to be skipped, use_yarn is set to False even if yarn
can be executed and reports that the yarn.lock file is valid.
"""
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.write_file('foo.js', 'fake contents')
self.write_file('yarn.lock', 'fake contents')
config = testutil.AppInfoFake(runtime='nodejs',
skip_files='^yarn\.lock$')
configurator = self.detect(appinfo=config)
self.assertEqual(configurator.data['use_yarn'], False)
self.generate_configs(appinfo=config, deploy=True)
self._validate_docker_files_for_npm()
self._validate_file_list_for_skip_yarn_lock()
def test_do_not_skip_yarn_lock(self):
"""Ensure use_yarn is True with yarn.lock present and not skipped.
"""
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.write_file('foo.js', 'fake contents')
self.write_file('yarn.lock', 'fake contents')
# Here only 'node_modules' will be skipped
config = testutil.AppInfoFake(runtime='nodejs',
skip_files='^node_modules$')
configurator = self.detect(appinfo=config)
self.assertEqual(configurator.data['use_yarn'], True)
self.generate_configs(appinfo=config, deploy=True)
self._validate_docker_files_for_yarn()
self._validate_file_list_for_skip_yarn_lock()
def test_use_yarn_skip_files_not_present(self):
"""Ensure use_yarn is True with yarn.lock present and not skipped.
In particular, this test ensures use_yarn is True even if app.yaml
doesn't contain a skip_files section.
"""
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.write_file('foo.js', 'fake contents')
self.write_file('yarn.lock', 'fake contents')
config = testutil.AppInfoFake(runtime='nodejs')
configurator = self.detect(appinfo=config)
self.assertEqual(configurator.data['use_yarn'], True)
self.generate_configs(appinfo=config, deploy=True)
self._validate_docker_files_for_yarn()
self._validate_file_list_for_skip_yarn_lock()
def test_node_js_package_json_no_write(self):
"""Test generate_config_data with a nodejs file and package.json."""
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
self.generate_configs()
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='nodejs'))
cfg_files = self.generate_config_data(deploy=True)
base_dockerfile = self.read_dist_file('data', 'Dockerfile')
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
base_dockerfile + 'COPY . /app/\n' +
self.read_dist_file('data', 'npm-package-json-install') +
'CMD npm start\n')
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.read_dist_file('data', 'dockerignore'))
self.assertEqual(set(os.listdir(self.temp_path)),
{'app.yaml', 'foo.js', 'package.json'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_detect_basic(self):
"""Ensure that appinfo will be generated in detect method."""
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
configurator = self.detect()
self.assertEqual(configurator.generated_appinfo,
{u'runtime': 'nodejs',
u'env': 'flex'})
def test_detect_custom(self):
"""Ensure that appinfo is correct with custom=True."""
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"start": "foo.js"}}')
configurator = self.detect(custom=True)
self.assertEqual(configurator.generated_appinfo,
{'runtime': 'custom',
'env': 'flex'})
def test_detect_no_start_no_server(self):
"""Ensure that detect fails if no scripts.start field, no server.js."""
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"not-start": "foo.js"}}')
configurator = self.detect()
self.assertEqual(configurator, None)
def test_detect_no_start_with_server(self):
"""Ensure appinfo generated if no scripts.start, server.js exists."""
self.write_file('server.js', 'bogus contents')
self.write_file('package.json', '{"scripts": {"not-start": "foo.js"}}')
configurator = self.detect()
self.assertEqual(configurator.generated_appinfo,
{'runtime': 'nodejs',
'env': 'flex'})
def test_node_js_with_engines(self):
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json',
'{"scripts": {"start": "foo.js"},'
'"engines": {"node": "0.12.3"}}')
self.generate_configs(deploy=True)
dockerfile_path = self.full_path('Dockerfile')
self.assertTrue(os.path.exists(dockerfile_path))
# This just verifies that the crazy node install line is generated, it
# says nothing about whether or not it works.
rx = re.compile(r'RUN npm install')
for line in open(dockerfile_path):
if rx.match(line):
break
else:
self.fail('node install line not generated')
def test_node_js_with_engines_no_write(self):
"""Test generate_config_data with 'engines' in package.json."""
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json',
'{"scripts": {"start": "foo.js"},'
'"engines": {"node": "0.12.3"}}')
cfg_files = self.generate_config_data(deploy=True)
self.assertIn('Dockerfile', [f.filename for f in cfg_files])
# This just verifies that the crazy node install line is generated, it
# says nothing about whether or not it works.
rx = re.compile(r'RUN npm install')
line_generated = False
for cfg_file in cfg_files:
if cfg_file.filename == 'Dockerfile':
for line in cfg_file.contents.split('\n'):
if rx.match(line):
line_generated = True
if not line_generated:
self.fail('node install line not generated')
def test_node_js_custom_runtime(self):
self.write_file('server.js', 'fake contents')
self.generate_configs(custom=True)
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='custom'))
self.assertEqual(sorted(os.listdir(self.temp_path)),
['.dockerignore', 'Dockerfile', 'app.yaml',
'server.js'])
def test_node_js_custom_runtime_no_write(self):
"""Test generate_config_data with custom runtime.
Should generate an app.yaml on disk, the Dockerfile and
.dockerignore in memory."""
self.write_file('server.js', 'fake contents')
cfg_files = self.generate_config_data(custom=True)
self.assert_file_exists_with_contents(
'app.yaml',
self.read_dist_file('data', 'app.yaml').format(runtime='custom'))
self.assertEqual(set(os.listdir(self.temp_path)),
{'app.yaml', 'server.js'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_node_js_runtime_field(self):
self.write_file('server.js', 'fake contents')
config = testutil.AppInfoFake(runtime='nodejs')
self.generate_configs(appinfo=config, deploy=True)
self.assertTrue(os.path.exists(self.full_path('Dockerfile')))
def test_node_js_custom_runtime_field(self):
self.write_file('server.js', 'fake contents')
config = testutil.AppInfoFake(runtime='custom')
self.assertTrue(self.generate_configs(appinfo=config, deploy=True))
def test_invalid_package_json(self):
self.write_file('package.json', '')
self.write_file('server.js', '')
self.generate_configs()
self.assertFalse(self.generate_configs())
# Tests that verify that the generated files match verbatim output.
# These will need to be maintained whenever the code generation changes,
# but this ensures that any diffs we introduce in the generate files will
# be reviewed.
def test_node_js_with_engines_retroactive(self):
self.write_file('foo.js', 'bogus contents')
self.write_file('package.json',
'{"scripts": {"start": "foo.js"},'
'"engines": {"node": "0.12.3"}}')
self.generate_configs(deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
textwrap.dedent("""\
# Dockerfile extending the generic Node image with application files for a
# single application.
FROM gcr.io/google_appengine/nodejs
# Check to see if the the version included in the base runtime satisfies
# 0.12.3, if not then do an npm install of the latest available
# version that satisfies it.
RUN /usr/local/bin/install_node 0.12.3
COPY . /app/
# You have to specify "--unsafe-perm" with npm install
# when running as root. Failing to do this can cause
# install to appear to succeed even if a preinstall
# script fails, and may have other adverse consequences
# as well.
# This command will also cat the npm-debug.log file after the
# build, if it exists.
RUN npm install --unsafe-perm || \\
((if [ -f npm-debug.log ]; then \\
cat npm-debug.log; \\
fi) && false)
CMD npm start
"""))
class FailureLoggingTests(testutil.TestBase):
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(FailureLoggingTests, self).setUp()
self.errors = []
self.debug = []
self.warnings = []
def error_fake(self, message):
self.errors.append(message)
def debug_fake(self, message):
self.debug.append(message)
def warn_fake(self, message):
self.warnings.append(message)
def test_invalid_package_json(self):
self.write_file('package.json', '')
self.write_file('server.js', '')
variations = [
(testutil.AppInfoFake(runtime='nodejs'), None),
(None, 'nodejs'),
(None, None)
]
for appinfo, runtime in variations:
self.errors = []
with mock.patch.dict(ext_runtime._LOG_FUNCS,
{'error': self.error_fake}):
self.generate_configs(appinfo=appinfo, runtime=runtime)
self.assertTrue(self.errors[0].startswith(
'node.js checker: error accessing package.json'))
def test_no_startup_script(self):
with mock.patch.dict(ext_runtime._LOG_FUNCS,
{'debug': self.debug_fake}):
self.generate_configs()
print self.debug
self.assertTrue(self.debug[1].startswith(
'node.js checker: Neither "start" in the "scripts" section '
'of "package.json" nor the "server.js" file were found.'))
variations = [
(testutil.AppInfoFake(runtime='nodejs'), None),
(None, 'nodejs')
]
for appinfo, runtime in variations:
self.errors = []
with mock.patch.dict(ext_runtime._LOG_FUNCS,
{'error': self.error_fake}):
self.generate_configs(appinfo=appinfo, runtime=runtime)
self.assertTrue(self.errors[0].startswith(
'node.js checker: Neither "start" in the "scripts" section '
'of "package.json" nor the "server.js" file were found.'))
def test_package_json_no_startup_script(self):
self.write_file('package.json', '{"scripts": {"not-start": "foo.js"}}')
variations = [
(testutil.AppInfoFake(runtime='nodejs'), None),
(None, 'nodejs'),
(None, None)
]
for appinfo, runtime in variations:
self.errors = []
with mock.patch.dict(ext_runtime._LOG_FUNCS,
{'error': self.error_fake}):
self.generate_configs(appinfo=appinfo, runtime=runtime)
self.assertTrue(self.errors[0].startswith(
'node.js checker: Neither "start" in the "scripts" section '
'of "package.json" nor the "server.js" file were found.'))
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,22 @@
Testing Notes
=============
To run the test script (`test/runtime_test.py`) you'll need to install the
Google Cloud SDK add this to your PYTHONPATH:
export PYTHONPATH=$CLOUD_SDK/lib:$CLOUD_SDK/platform/google_appengine:$PYTHONPATH
Where `$CLOUD_SDK` is the location where the cloud SDK is installed.
Debugging
---------
If you do "sys.stderr.write(...)" from a plugin script, your output will go
to a nice shiny bright warning message. Exceptions will show up as warning
messages simply by virtue of having been formatted to standard error.
Full logging can be made to happen by adding:
from googlecloudsdk.core import log
log.SetVerbosity(logging.DEBUG)

View File

@@ -0,0 +1,24 @@
Want to contribute? Great! First, read this page (including the small print at the end).
### Before you contribute
Before we can use your code, you must sign the
[Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1)
(CLA), which you can do online. The CLA is necessary mainly because you own the
copyright to your changes, even after your contribution becomes part of our
codebase, so we need your permission to use and distribute your code. We also
need to be sure of various other things—for instance that you'll tell us if you
know that your code infringes on other people's patents. You don't have to sign
the CLA until after you've submitted your code for review and a member has
approved it, but you must do it before we can put your code into our codebase.
Before you start working on a larger contribution, you should get in touch with
us first through the issue tracker with your idea so that we can help out and
possibly guide you. Coordinating up front makes it much easier to avoid
frustration later on.
### Code reviews
All submissions, including submissions by project members, require review. We
use Github pull requests for this purpose.
### The small print
Contributions made by corporations are covered by a different agreement than
the one above, the Software Grant and Corporate Contributor License Agreement.

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016 Google Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,25 @@
PHP Externalized Runtime
============================
This is the PHP runtime definition that is used by gcloud to generate
Dockerfiles for GAE.
How to Run Tests
============================
In order to test your runtime changes you will need to run the following
commands.
```
$ sudo apt-get install python-pip
$ sudo pip install virtualenv
$ cd $CODE_DIRECTORY
$ git clone $GIT_HOST/ext-runtime-sdk
$ virtualenv test_env
$ . test_env/bin/activate
(test_env)$ pip install ext-runtime-sdk/
(test_env)$ pip install PyYaml appengine-config-transformer
(test_env)$ cd gae-xrt-php
(test_env)$ python test/runtime_test.py
```

View File

@@ -0,0 +1,90 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# PHP detection script.
import json
import os
import posixpath
import sys
def get_config():
"""Requests the current runtime configuration.
Returns:
A dictionary containing the current runtime configuration.
Raises:
IOError: An error occurred writing to the controller.
"""
json.dump({'type': 'get_config'}, sys.stdout)
sys.stdout.write('\n')
sys.stdout.flush()
return json.loads(sys.stdin.readline())
def main(args):
if len(args) != 2:
print 'Invalid Usage: {} <source-root-directory>'.format(args[0])
return 1
config = get_config()
params = config['params']
appinfo = params['appinfo']
entrypoint = None
if appinfo and 'entrypoint' in appinfo:
entrypoint = appinfo['entrypoint']
if isinstance(entrypoint, list):
entrypoint = json.dumps(entrypoint)
root = '.'
document_root = '/app'
if appinfo and 'runtime_config' in appinfo:
if 'document_root' in appinfo['runtime_config']:
root = appinfo['runtime_config']['document_root']
document_root = posixpath.normpath(posixpath.join('/app', root))
directory = args[1]
for dirpath, _, filenames in os.walk(directory):
for filename in filenames:
if filename.endswith('.php'):
# "appinfo" writes to the app.yaml when it's generated, and
# "runtime_data" writes to the Dockerfile
json_config = {
'type': 'runtime_parameters',
'appinfo': {
'runtime': 'custom' if params['custom'] else 'php',
'env': 'flex',
'runtime_config': {
'document_root': root,
},
},
'runtime_data': {
'entrypoint': entrypoint,
'document_root': document_root,
},
}
# Prevent "entrypoint: null" in generated app.yaml
#if entrypoint:
# json_config['appinfo']['entrypoint'] = entrypoint
json.dump(json_config, sys.stdout)
sys.stdout.write('\n')
sys.stdout.flush()
return 0
return 1
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,84 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import os
import sys
def get_config():
"""Requests the current runtime configuration.
Returns:
A dictionary containing the current runtime configuration.
Raises:
IOError: An error occurred writing to the controller.
"""
json.dump({'type': 'get_config'}, sys.stdout)
sys.stdout.write('\n')
sys.stdout.flush()
return json.loads(sys.stdin.readline())
def send_file(filename, data):
"""Sends file data to be saved by the controller.
send_file currently only supports sending ASCII-encoded data. For binary
data it must be Base64 encoded and additional "encoding" parameter must be
sent indicating the data has been encoded.
Args:
filename: (str) Target filename.
data: (str) File contents.
Raises:
IOError: An error occurred writing to the controller.
"""
json.dump({
'type': 'gen_file',
'filename': filename,
'contents': data}, sys.stdout)
sys.stdout.write('\n')
sys.stdout.flush()
def main(args):
config = get_config()
params = config['params']
if not params['custom'] and not params['deploy']:
return 0
# Locate template directory.
template_dir = os.path.join(os.path.dirname(__file__), '..', 'templates')
# Generate Dockerfile
entrypoint = config['runtime_data']['entrypoint']
template_name = 'Dockerfile.entrypoint.template' if entrypoint else 'Dockerfile.template'
path = os.path.join(template_dir, template_name)
with open(path, 'r') as f:
data = f.read()
document_root = config['runtime_data']['document_root']
send_file('Dockerfile', data.format(document_root=document_root,
entrypoint=entrypoint))
# Generate .dockerignore
path = os.path.join(template_dir, 'dockerignore.template')
with open(path, 'r') as f:
send_file('.dockerignore', f.read())
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,8 @@
author: adamtanner@google.com
name: php
description: Official Google PHP runtime definition.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,10 @@
# Dockerfile extending the generic PHP image with application files for a
# single application.
FROM gcr.io/google-appengine/php:latest
# The Docker image will configure the document root according to this
# environment variable.
ENV DOCUMENT_ROOT {document_root}
# Allow custom CMD
CMD {entrypoint}

View File

@@ -0,0 +1,7 @@
# Dockerfile extending the generic PHP image with application files for a
# single application.
FROM gcr.io/google-appengine/php:latest
# The Docker image will configure the document root according to this
# environment variable.
ENV DOCUMENT_ROOT {document_root}

View File

@@ -0,0 +1,19 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
.dockerignore
Dockerfile
.git
.hg
.svn

View File

@@ -0,0 +1,279 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import textwrap
import unittest
from gae_ext_runtime import testutil
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
class RuntimeTestCase(testutil.TestBase):
"""Tests for the PHP external runtime fingerprinter."""
def license(self):
return textwrap.dedent('''\
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
''')
def preamble(self):
return textwrap.dedent('''\
# Dockerfile extending the generic PHP image with application files for a
# single application.
FROM gcr.io/google-appengine/php:latest
# The Docker image will configure the document root according to this
# environment variable.
''')
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTestCase, self).setUp()
def file_contents(self, filename):
with open(self.full_path(filename)) as f:
return f.read()
def test_generate_without_php_files(self):
self.write_file('index.html', 'index')
self.assertFalse(self.generate_configs())
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
self.assertFalse(os.path.exists(self.full_path('Dockerfile')))
self.assertFalse(os.path.exists(self.full_path('.dockerignore')))
def test_generate_with_php_files(self):
self.write_file('index.php', 'index')
self.generate_configs()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: php\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('runtime_config:\n document_root: .\n', app_yaml)
self.assertNotIn('entrypoint', app_yaml)
self.assertFalse(os.path.exists(self.full_path('Dockerfile')))
self.assertFalse(os.path.exists(self.full_path('.dockerignore')))
def test_generate_with_php_files_no_write(self):
"""Test generate_config_data with a .php file.
Checks app.yaml contents, app.yaml is written to disk, and
Dockerfile and .dockerignore not in the directory.
"""
self.write_file('index.php', 'index')
self.generate_config_data()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: php\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('runtime_config:\n document_root: .\n', app_yaml)
self.assertFalse(os.path.exists(self.full_path('Dockerfile')))
self.assertFalse(os.path.exists(self.full_path('.dockerignore')))
def test_generate_custom_runtime(self):
self.write_file('index.php', 'index')
self.generate_configs(custom=True)
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(dockerfile, self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app
'''))
self.assert_file_exists_with_contents(
'.dockerignore',
self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_custom_runtime_no_write(self):
"""Tests generate_config_data with custom runtime."""
self.write_file('index.php', 'index')
cfg_files = self.generate_config_data(custom=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app
'''))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_with_deploy(self):
self.write_file('index.php', 'index')
self.generate_configs(deploy=True)
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(dockerfile, textwrap.dedent('''\
# Dockerfile extending the generic PHP image with application files for a
# single application.
FROM gcr.io/google-appengine/php:latest
# The Docker image will configure the document root according to this
# environment variable.
ENV DOCUMENT_ROOT /app
'''))
dockerignore = self.file_contents('.dockerignore')
self.assertEqual(dockerignore, self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_with_deploy_no_write(self):
"""Tests generate_config_data with deploy=True."""
self.write_file('index.php', 'index')
cfg_files = self.generate_config_data(deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app
'''))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_with_existing_appinfo(self):
self.write_file('index.php', 'index')
appinfo = testutil.AppInfoFake(
runtime_config={'document_root': 'wordpress'},
entrypoint='["/bin/bash", "my-cmd.sh"]')
self.generate_configs(deploy=True, appinfo=appinfo)
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(dockerfile, self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app/wordpress
# Allow custom CMD
CMD ["/bin/bash", "my-cmd.sh"]
'''))
dockerignore = self.file_contents('.dockerignore')
self.assertEqual(dockerignore, self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_with_existing_appinfo_no_write(self):
"""Tests generate_config_data with fake appinfo."""
self.write_file('index.php', 'index')
appinfo = testutil.AppInfoFake(
runtime_config={'document_root': 'wordpress'},
entrypoint='["/bin/bash", "my-cmd.sh"]')
cfg_files = self.generate_config_data(deploy=True, appinfo=appinfo)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app/wordpress
# Allow custom CMD
CMD ["/bin/bash", "my-cmd.sh"]
'''))
self.assert_genfile_exists_with_contents(
cfg_files,
'.dockerignore',
self.license() + textwrap.dedent('''\
.dockerignore
Dockerfile
.git
.hg
.svn
'''))
def test_generate_with_array_entrypoint(self):
self.write_file('index.php', 'index')
appinfo = testutil.AppInfoFake(
runtime_config={'document_root': 'wordpress'},
entrypoint=['/bin/bash', 'my-cmd.sh'])
self.generate_configs(deploy=True, appinfo=appinfo)
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(dockerfile, self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app/wordpress
# Allow custom CMD
CMD ["/bin/bash", "my-cmd.sh"]
'''))
def test_generate_with_array_entrypoint_no_write(self):
"""Tests generate_config_data with an array entrypoint."""
self.write_file('index.php', 'index')
appinfo = testutil.AppInfoFake(
runtime_config={'document_root': 'wordpress'},
entrypoint=["/bin/bash", "my-cmd.sh"])
cfg_files = self.generate_config_data(deploy=True, appinfo=appinfo)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.preamble() + textwrap.dedent('''\
ENV DOCUMENT_ROOT /app/wordpress
# Allow custom CMD
CMD ["/bin/bash", "my-cmd.sh"]
'''))
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,24 @@
Want to contribute? Great! First, read this page (including the small print at the end).
### Before you contribute
Before we can use your code, you must sign the
[Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1)
(CLA), which you can do online. The CLA is necessary mainly because you own the
copyright to your changes, even after your contribution becomes part of our
codebase, so we need your permission to use and distribute your code. We also
need to be sure of various other things—for instance that you'll tell us if you
know that your code infringes on other people's patents. You don't have to sign
the CLA until after you've submitted your code for review and a member has
approved it, but you must do it before we can put your code into our codebase.
Before you start working on a larger contribution, you should get in touch with
us first through the issue tracker with your idea so that we can help out and
possibly guide you. Coordinating up front makes it much easier to avoid
frustration later on.
### Code reviews
All submissions, including submissions by project members, require review. We
use Github pull requests for this purpose.
### The small print
Contributions made by corporations are covered by a different agreement than
the one above, the Software Grant and Corporate Contributor License Agreement.

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,6 @@
Python Externalized Runtime
===========================
This is the python runtime definition that will soon be used by gcloud for
generating dockerfiles for GAE.

View File

@@ -0,0 +1,129 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Language detection script.
import json
import os
import sys
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
import versions
def PythonVersionFromRuntimeConfig(runtime_config):
"""Validate the python_version section of runtime_config, if present.
Valid values are set in versions.PYTHON_INTERPRETER_VERSION_MAP. If
runtime_config is false-equivalent or if the python_version field is
absent, versions.DEFAULT_PYTHON_INTERPRETER_VERSION is used as the
default.
Args:
runtime_config: A valid runtime_config.
Returns:
One of the strings from versions.PYTHON_INTERPRETER_VERSION_MAP.keys().
Raises:
ValueError: If the contents of the python_version field are not valid.
"""
if not runtime_config:
python_version = None
else:
python_version = runtime_config.python_version
if not python_version:
return versions.DEFAULT_PYTHON_INTERPRETER_VERSION
elif python_version in versions.PYTHON_INTERPRETER_VERSION_MAP:
return python_version
else:
raise ValueError('Unsupported or invalid python version specified.')
def Main(args):
if len(args) != 2:
# If we're being called incorrectly, this probably isn't happening from a
# framework.
sys.stderr.write('Invalid Usage: %s <source-root-directory>' % sys.argv[0])
return 1
# Get the first argument, should be the source root directory.
path = args[1]
# See if we have a config file and if it contains an entrypoint.
config = comm.get_config()
appinfo = config.params.appinfo
entrypoint = None
if appinfo:
entrypoint = appinfo.entrypoint
comm.info('Checking for Python.')
got_shrinkwrap = False
requirements_txt = os.path.join(path, 'requirements.txt')
got_requirements_txt = os.path.isfile(requirements_txt)
got_py_files = False
# check for any python files.
for _, _, files in os.walk(path):
for filename in files:
if filename.endswith('.py'):
got_py_files = True
if not got_requirements_txt and not got_py_files:
return 1
# Query the user for the WSGI entrypoint:
if not entrypoint:
entrypoint = comm.query_user('This looks like a Python app. If so, please '
'enter the command to run the app in '
"production (enter nothing if it's not a "
'python app): ')
if not entrypoint:
comm.info('No entrypoint specified. Assuming this is not a python app.')
return 1
elif appinfo:
comm.print_status('To avoid being asked for an entrypoint in the '
'future, please add the entrypoint to your app.yaml:\n'
' entrypoint: %s' % entrypoint)
try:
# Get the python interpreter version. Use the default if not specified.
python_version = PythonVersionFromRuntimeConfig(
appinfo.runtime_config if appinfo else None)
except ValueError:
# The python version was selected, but set to an invalid result.
valid_versions = str(sorted(versions.PYTHON_INTERPRETER_VERSION_MAP.keys()))
comm.error('The python_version selected in runtime_config is invalid or not '
'supported. Please select from the following options:\n'
'%s', valid_versions)
return 1
runtime = 'custom' if config.params.custom else 'python'
comm.send_runtime_params({'entrypoint': entrypoint,
'got_requirements_txt': got_requirements_txt,
'python_version': python_version},
appinfo={'runtime': runtime,
'entrypoint': entrypoint,
'env': 'flex'})
return 0
if __name__ == '__main__':
sys.exit(Main(sys.argv))

View File

@@ -0,0 +1,60 @@
#!/usr/bin/python
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import StringIO
import sys
# Augment the path with our library directory.
ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0])))
sys.path.append(os.path.join(ROOT_DIR, 'lib'))
import comm
import versions
ROOT = comm.RuntimeDefinitionRoot(ROOT_DIR)
def Main(args):
"""Generate all config files for the module."""
# Get runtime parameters from the controller.
config = comm.get_config()
params = config.params
runtime_data = config.runtime_data
# Customize the dockerfile.
if params.custom or params.deploy:
# Generate the Dockerfile.
out = StringIO.StringIO()
out.write(ROOT.read_file('data', 'Dockerfile.preamble'))
docker_python_version = versions.PYTHON_INTERPRETER_VERSION_MAP[
runtime_data.python_version]
out.write(ROOT.read_file('data', 'Dockerfile.virtualenv.template').format(
python_version=docker_python_version))
if runtime_data.got_requirements_txt:
out.write(ROOT.read_file('data', 'Dockerfile.requirements_txt'))
out.write(ROOT.read_file('data', 'Dockerfile.install_app'))
if runtime_data.entrypoint:
out.write('CMD {0}\n'.format(runtime_data.entrypoint))
comm.gen_file('Dockerfile', out.getvalue())
# Generate .dockerignore
comm.gen_file('.dockerignore', ROOT.read_file('data', 'dockerignore'))
if __name__ == '__main__':
Main(sys.argv)

View File

@@ -0,0 +1 @@
FROM gcr.io/google-appengine/python

View File

@@ -0,0 +1,2 @@
ADD requirements.txt /app/
RUN pip install -r requirements.txt

View File

@@ -0,0 +1,7 @@
LABEL python_version=python{python_version}
RUN virtualenv --no-download /env -p python{python_version}
# Set virtualenv environment variables. This is equivalent to running
# source /env/bin/activate
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

View File

@@ -0,0 +1,3 @@
runtime: {runtime}
env: flex
entrypoint: {entrypoint}

View File

@@ -0,0 +1,19 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
.dockerignore
Dockerfile
.git
.hg
.svn

View File

@@ -0,0 +1,174 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Note: this file is part of the sdk-ext-runtime package. It gets copied into
# individual GAE runtime modules so that they can be easily deployed.
import json
import os
import sys
class JSONObject(object):
"""Wrapper for a JSON object.
Presents a JSON object as a python object (where fields are attributes)
instead of a dictionary. Undefined attributes produce a value of None
instead of an AttributeError.
Note that attribute names beginning with an underscore are excluded.
"""
def __getattr__(self, attr):
return None
def to_dict(self):
result = {}
for attr, val in self.__dict__.iteritems():
if not attr.startswith('_'):
result[attr] = _make_serializable(val)
return result
# Alias old style naming so this interoperates with gcloud's appinfo.
ToDict = to_dict
def _make_serializable(obj):
"""Converts objects to serializable form."""
if isinstance(obj, JSONObject):
return obj.to_dict()
else:
return obj
def _write_msg(**message):
"""Write a message to standard output.
Args:
**message: ({str: object, ...}) A JSON message encoded in keyword
arguments.
"""
json.dump(message, sys.stdout, default=_make_serializable)
sys.stdout.write('\n')
sys.stdout.flush()
def error(message, *args):
_write_msg(type='error', message=message % args)
def warn(message, *args):
_write_msg(type='warn', message=message % args)
def info(message, *args):
_write_msg(type='info', message=message % args)
def debug(message, *args):
_write_msg(type='debug', message=message % args)
def print_status(message, *args):
_write_msg(type='print_status', message=message % args)
def send_runtime_params(params, appinfo=None):
"""Send runtime parameters back to the controller.
Args:
params: ({str: object, ...}) Set of runtime parameters. Must be
json-encodable.
appinfo: ({str: object, ...} or None) Contents of the app.yaml file to
be produced by the runtime definition. Required fields may be
added to this by the framework, the only thing an application
needs to provide is the "runtime" field and any additional data
fields.
"""
if appinfo is not None:
_write_msg(type='runtime_parameters', runtime_data=params,
appinfo=appinfo)
else:
_write_msg(type='runtime_parameters', runtime_data=params)
def get_config():
"""Request runtime parameters from the controller.
Returns:
(object) The runtime parameters represented as an object.
"""
_write_msg(type='get_config')
return dict_to_object(json.loads(sys.stdin.readline()))
def dict_to_object(json_dict):
"""Converts a dictionary to a python object.
Converts key-values to attribute-values.
Args:
json_dict: ({str: object, ...})
Returns:
(JSONObject)
"""
obj = JSONObject()
for name, val in json_dict.iteritems():
if isinstance(val, dict):
val = dict_to_object(val)
setattr(obj, name, val)
return obj
class RuntimeDefinitionRoot(object):
"""Abstraction that allows us to access files in the runtime definiton."""
def __init__(self, path):
self.root = path
def read_file(self, *name):
with open(os.path.join(self.root, *name)) as src:
return src.read()
def gen_file(name, contents):
"""Generate the file.
This writes the file to be generated back to the controller.
Args:
name: (str) The UNIX-style relative path of the file.
contents: (str) The complete file contents.
"""
_write_msg(type='gen_file', filename=name, contents=contents)
def query_user(prompt, default=None):
"""Query the user for data.
Args:
prompt: (str) Prompt to display to the user.
default: (str or None) Default value to use if the user doesn't input
anything.
Returns:
(str) Value returned by the user.
"""
kwargs = {}
kwargs['prompt'] = prompt
if default is not None:
kwargs['default'] = default
_write_msg(type='query_user', **kwargs)
return json.loads(sys.stdin.readline())['result']

View File

@@ -0,0 +1,28 @@
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Table of supported Python interpreter versions."""
# Missing python_version defaults to this
DEFAULT_PYTHON_INTERPRETER_VERSION = '2'
# Map from app.yaml "python_version" to Dockerfile.virtualenv.template
# {python_version}
PYTHON_INTERPRETER_VERSION_MAP = {
'2': '',
'3': '3.6',
'3.4': '3.4',
'3.5': '3.5',
'3.6': '3.6',
}

View File

@@ -0,0 +1,8 @@
author: mmuller@google.com
name: python
description: Official Google Python runtime definition.
api_version: 0.1
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,449 @@
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Note: this file is part of the sdk-ext-runtime package. It gets copied into
# individual GAE runtime modules so that they can be easily deployed.
import os
import textwrap
import unittest
from gae_ext_runtime import comm
from gae_ext_runtime import ext_runtime
from gae_ext_runtime import testutil
ROOT_DIR = os.path.dirname(os.path.dirname(__file__))
ROOT = comm.RuntimeDefinitionRoot(ROOT_DIR)
class FakeExecutionEnvironment(ext_runtime.DefaultExecutionEnvironment):
def CanPrompt(self):
return True
def PromptResponse(self, message):
return 'my_entrypoint'
class RuntimeTests(testutil.TestBase):
def setUp(self):
# This has to come before RuntimeTests.setUp()
self.runtime_def_root = ROOT_DIR
super(RuntimeTests, self).setUp()
self.DOCKERFILE_PREAMBLE = (
ROOT.read_file('data', 'Dockerfile.preamble'))
self.DOCKERFILE_VIRTUALENV_TEMPLATE = (
ROOT.read_file('data',
'Dockerfile.virtualenv.template'))
self.DOCKERFILE_REQUIREMENTS_TXT = (
ROOT.read_file('data', 'Dockerfile.requirements_txt'))
self.DOCKERFILE_INSTALL_APP = (
ROOT.read_file('data', 'Dockerfile.install_app'))
self.set_execution_environment(FakeExecutionEnvironment())
# XXX Move this to testutil.
def read_file(self, *args):
"""Read the file, return the contents.
Args:
*args: A set of path components (see full_path()) relative to the
temporary directory.
"""
with open(os.path.join(self.temp_path, *args)) as fp:
return fp.read()
def test_python(self):
self.write_file('requirements.txt', 'requirements')
self.generate_configs(deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(python_version='') +
self.DOCKERFILE_REQUIREMENTS_TXT +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'requirements.txt',
'app.yaml',
'Dockerfile',
'.dockerignore'})
def test_python_no_write(self):
"""Tests generate_config_data with only requirements.txt.
app.yaml should be written to disk, Dockerfile and .dockerignore
returned by the method in memory. Tests that Dockerfile contents
are correct.
"""
self.write_file('requirements.txt', 'requirements')
cfg_files = self.generate_config_data(deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(python_version='') +
self.DOCKERFILE_REQUIREMENTS_TXT +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'requirements.txt', 'app.yaml'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_no_requirements_txt(self):
self.write_file('foo.py', '# python code')
self.generate_configs(custom=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'foo.py', 'app.yaml', 'Dockerfile', '.dockerignore'})
def test_python_no_requirements_txt_no_write(self):
"""Tests generate_config_data with no requirements.txt file."""
self.write_file('foo.py', '# python code')
cfg_files = self.generate_config_data(custom=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'foo.py', 'app.yaml'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_with_app_yaml(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!')
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', 'Dockerfile', '.dockerignore'})
def test_python_with_app_yaml_no_write(self):
"""Tests generate_config_data with fake appinfo."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!')
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_app_yaml_no_entrypoint(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python')
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', 'Dockerfile', '.dockerignore'})
def test_python_app_yaml_no_entrypoint_no_write(self):
"""Tests generate_config_data with fake appinfo, no entrypoint."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python')
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD my_entrypoint\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_with_runtime_config_but_no_python_version(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!')
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_runtime_config_but_no_python_version_no_write(self):
"""Tests generate_config_data with no python version in appinfo."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!')
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_with_explicit_python2(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='2'))
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_explicit_python2_no_write(self):
"""Tests generate_config_data with Python version '2' in appinfo."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='2'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_with_explicit_python3(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3'))
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='3.6') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_explicit_python3_no_write(self):
"""Tests generate_config_data with python version '3' in appinfo."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='3.6') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_with_explicit_python34(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3.4'))
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='3.4') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_explicit_python35(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3.5'))
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='3.5') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_explicit_python36(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3.6'))
self.generate_configs(appinfo=config, deploy=True)
self.assert_file_exists_with_contents(
'Dockerfile',
self.DOCKERFILE_PREAMBLE +
self.DOCKERFILE_VIRTUALENV_TEMPLATE.format(
python_version='3.6') +
self.DOCKERFILE_INSTALL_APP +
'CMD run_me_some_python!\n')
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', '.dockerignore', 'Dockerfile'})
def test_python_with_invalid_version(self):
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='invalid_version'))
self.assertRaises(testutil.InvalidRuntime,
self.generate_config_data, appinfo=config)
def test_python_custom_runtime(self):
self.write_file('test.py', 'test file')
self.generate_configs(custom=True)
with open(os.path.join(self.temp_path, 'app.yaml')) as f:
app_yaml_contents = f.read()
self.assertMultiLineEqual(
app_yaml_contents,
textwrap.dedent("""\
entrypoint: my_entrypoint
env: flex
runtime: custom
"""))
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', 'app.yaml', '.dockerignore', 'Dockerfile'})
def test_python_custom_runtime_no_write(self):
"""Tests generate_config_data with custom=True."""
self.write_file('test.py', 'test file')
cfg_files = self.generate_config_data(custom=True)
with open(os.path.join(self.temp_path, 'app.yaml')) as f:
app_yaml_contents = f.read()
self.assertMultiLineEqual(
app_yaml_contents,
textwrap.dedent("""\
entrypoint: my_entrypoint
env: flex
runtime: custom
"""))
self.assertEqual(set(os.listdir(self.temp_path)),
{'test.py', 'app.yaml'})
self.assertEqual({f.filename for f in cfg_files},
{'Dockerfile', '.dockerignore'})
def test_python_custom_runtime_field(self):
"""Verify that a runtime field of "custom" works."""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(runtime='custom',
entrypoint='my_entrypoint')
self.assertTrue(self.generate_configs(appinfo=config, deploy=True))
def test_python_label(self):
"""Test that a LABEL directive is added to the Dockerfile"""
self.write_file('test.py', 'test file')
config = testutil.AppInfoFake(
runtime='python',
entrypoint='run_me_some_python!',
runtime_config=dict(python_version='3'))
cfg_files = self.generate_config_data(appinfo=config, deploy=True)
dockerfile = [f for f in cfg_files if f.filename == 'Dockerfile'][0]
self.assertIn('LABEL python_version=python3.6\n', dockerfile.contents)
# NOTE: this test is also irrelevant to the runtime, convert it to
# something appropriate to the framework.
# def test_python_non_interactive(self):
# self.write_file('test.py', 'blah')
# with mock.patch.object(console_io, 'CanPrompt', lambda: False):
# with mock.patch.object(fingerprinting,
# 'GetNonInteractiveErrorMessage',
# lambda: 'xx123unlikely'):
#
# fingerprinter.IdentifyDirectory(self.temp_path)
# self.AssertLogContains('xx123unlikely')
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,24 @@
Want to contribute? Great! First, read this page (including the small print at the end).
### Before you contribute
Before we can use your code, you must sign the
[Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1)
(CLA), which you can do online. The CLA is necessary mainly because you own the
copyright to your changes, even after your contribution becomes part of our
codebase, so we need your permission to use and distribute your code. We also
need to be sure of various other things—for instance that you'll tell us if you
know that your code infringes on other people's patents. You don't have to sign
the CLA until after you've submitted your code for review and a member has
approved it, but you must do it before we can put your code into our codebase.
Before you start working on a larger contribution, you should get in touch with
us first through the issue tracker with your idea so that we can help out and
possibly guide you. Coordinating up front makes it much easier to avoid
frustration later on.
### Code reviews
All submissions, including submissions by project members, require review. We
use Github pull requests for this purpose.
### The small print
Contributions made by corporations are covered by a different agreement than
the one above, the Software Grant and Corporate Contributor License Agreement.

View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016 Google Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

View File

@@ -0,0 +1,25 @@
Ruby Externalized Runtime
============================
This is the Ruby runtime definition that is used by gcloud to generate
Dockerfiles for GAE.
How to Run Tests
============================
In order to test your runtime changes you will need to run the following
commands.
```
$ sudo apt-get install python-pip
$ sudo pip install virtualenv
$ cd $CODE_DIRECTORY
$ git clone $GIT_HOST/ext-runtime-sdk
$ virtualenv test_env
$ . test_env/bin/activate
(test_env)$ pip install ext-runtime-sdk/
(test_env)$ pip install PyYaml appengine-config-transformer
(test_env)$ cd gae-xrt-ruby
(test_env)$ python test/runtime_test.py
```

View File

@@ -0,0 +1,81 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Ruby detection script.
import glob
import json
import os
import sys
# HACK: Import a vendored gae_ext_runtime library until we have a better
# solution.
sys.path.append(os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0]))), 'lib'))
from gae_ext_runtime import comm
def main(args):
if len(args) != 2:
print 'Invalid Usage: {} <source-root-directory>'.format(args[0])
return 1
directory = args[1]
# The directory is considered a Ruby application if any are true:
# * The top-level directory contains a Gemfile
# * The top-level directory contains a config.ru
# * The top-level directory contains a file ending in .rb
gemfile = os.path.isfile(os.path.join(directory, 'Gemfile'))
configru = os.path.isfile(os.path.join(directory, 'config.ru'))
ruby_file = glob.glob(os.path.join(directory, '*.rb'))
if not gemfile and not configru and not ruby_file:
return 1
try:
with open(os.path.join(directory, '.ruby-version')) as f:
ruby_version = f.read().strip()
except IOError:
ruby_version = ''
config = comm.get_config()
appinfo = config.params.appinfo
if appinfo and appinfo.entrypoint:
entrypoint = appinfo.entrypoint
else:
default = 'bundle exec rackup -p $PORT'
prompt = ('This looks like a Ruby application. Please confirm the '
'command to run as the entrypoint:')
# Currently query_user does not seem to return the default on empty
# response, so do the defaulting ourselves until that is fixed.
entrypoint = comm.query_user(prompt, default) or default
runtime_data = {
'ruby_version': ruby_version,
'entrypoint': entrypoint,
}
appinfo = {
'runtime': 'custom' if config.params.custom else 'ruby',
'env': 'flex',
'entrypoint': entrypoint,
}
comm.send_runtime_params(runtime_data, appinfo=appinfo)
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,56 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import os
import sys
# HACK: Import a vendored gae_ext_runtime library until we have a better
# solution.
sys.path.append(os.path.join(
os.path.dirname(os.path.dirname(os.path.abspath(sys.argv[0]))), 'lib'))
from gae_ext_runtime import comm
def main(args):
config = comm.get_config()
params = config.params
if not params.custom and not params.deploy:
return 0
entrypoint = config.runtime_data.entrypoint
ruby_version = config.runtime_data.ruby_version
base_image_tag = os.environ.get('GAE_RUBY_BASE_IMAGE_TAG') or 'latest'
# Locate template directory.
template_dir = os.path.join(os.path.dirname(__file__), '..', 'templates')
# Generate Dockerfile
path = os.path.join(template_dir, 'Dockerfile.template')
with open(path, 'r') as f:
data = f.read()
comm.gen_file('Dockerfile', data.format(ruby_version=ruby_version,
entrypoint=entrypoint, base_image_tag=base_image_tag))
# Generate .dockerignore
path = os.path.join(template_dir, 'dockerignore.template')
with open(path, 'r') as f:
comm.gen_file('.dockerignore', f.read())
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

View File

@@ -0,0 +1,174 @@
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Note: this file is part of the sdk-ext-runtime package. It gets copied into
# individual GAE runtime modules so that they can be easily deployed.
import json
import os
import sys
class JSONObject(object):
"""Wrapper for a JSON object.
Presents a JSON object as a python object (where fields are attributes)
instead of a dictionary. Undefined attributes produce a value of None
instead of an AttributeError.
Note that attribute names beginning with an underscore are excluded.
"""
def __getattr__(self, attr):
return None
def to_dict(self):
result = {}
for attr, val in self.__dict__.iteritems():
if not attr.startswith('_'):
result[attr] = _make_serializable(val)
return result
# Alias old style naming so this interoperates with gcloud's appinfo.
ToDict = to_dict
def _make_serializable(obj):
"""Converts objects to serializable form."""
if isinstance(obj, JSONObject):
return obj.to_dict()
else:
return obj
def _write_msg(**message):
"""Write a message to standard output.
Args:
**message: ({str: object, ...}) A JSON message encoded in keyword
arguments.
"""
json.dump(message, sys.stdout, default=_make_serializable)
sys.stdout.write('\n')
sys.stdout.flush()
def error(message, *args):
_write_msg(type='error', message=message % args)
def warn(message, *args):
_write_msg(type='warn', message=message % args)
def info(message, *args):
_write_msg(type='info', message=message % args)
def debug(message, *args):
_write_msg(type='debug', message=message % args)
def print_status(message, *args):
_write_msg(type='print_status', message=message % args)
def send_runtime_params(params, appinfo=None):
"""Send runtime parameters back to the controller.
Args:
params: ({str: object, ...}) Set of runtime parameters. Must be
json-encodable.
appinfo: ({str: object, ...} or None) Contents of the app.yaml file to
be produced by the runtime definition. Required fields may be
added to this by the framework, the only thing an application
needs to provide is the "runtime" field and any additional data
fields.
"""
if appinfo is not None:
_write_msg(type='runtime_parameters', runtime_data=params,
appinfo=appinfo)
else:
_write_msg(type='runtime_parameters', runtime_data=params)
def get_config():
"""Request runtime parameters from the controller.
Returns:
(object) The runtime parameters represented as an object.
"""
_write_msg(type='get_config')
return dict_to_object(json.loads(sys.stdin.readline()))
def dict_to_object(json_dict):
"""Converts a dictionary to a python object.
Converts key-values to attribute-values.
Args:
json_dict: ({str: object, ...})
Returns:
(JSONObject)
"""
obj = JSONObject()
for name, val in json_dict.iteritems():
if isinstance(val, dict):
val = dict_to_object(val)
setattr(obj, name, val)
return obj
class RuntimeDefinitionRoot(object):
"""Abstraction that allows us to access files in the runtime definiton."""
def __init__(self, path):
self.root = path
def read_file(self, *name):
with open(os.path.join(self.root, *name)) as src:
return src.read()
def gen_file(name, contents):
"""Generate the file.
This writes the file to be generated back to the controller.
Args:
name: (str) The UNIX-style relative path of the file.
contents: (str) The complete file contents.
"""
_write_msg(type='gen_file', filename=name, contents=contents)
def query_user(prompt, default=None):
"""Query the user for data.
Args:
prompt: (str) Prompt to display to the user.
default: (str or None) Default value to use if the user doesn't input
anything.
Returns:
(str) Value returned by the user.
"""
kwargs = {}
kwargs['prompt'] = prompt
if default is not None:
kwargs['default'] = default
_write_msg(type='query_user', **kwargs)
return json.loads(sys.stdin.readline())['result']

View File

@@ -0,0 +1,8 @@
author: adamtanner@google.com
name: ruby
description: Official Google Ruby runtime definition.
api_version: 0.2
detect:
python: bin/detect
generate_configs:
python: bin/generate_configs

View File

@@ -0,0 +1,54 @@
# This Dockerfile for a Ruby application was generated by gcloud.
# The base Dockerfile installs:
# * A number of packages needed by the Ruby runtime and by gems
# commonly used in Ruby web apps (such as libsqlite3)
# * A recent version of NodeJS
# * A recent version of the standard Ruby runtime to use by default
# * The bundler gem
FROM gcr.io/google-appengine/ruby:{base_image_tag}
# If your application requires a specific ruby version (compatible with rbenv),
# set it here. Leave blank to use the currently recommended default.
ARG REQUESTED_RUBY_VERSION="{ruby_version}"
# Install any requested ruby if not already preinstalled by the base image.
# Tries installing a prebuilt package first, then falls back to a source build.
RUN if test -n "$REQUESTED_RUBY_VERSION" -a \
! -x /rbenv/versions/$REQUESTED_RUBY_VERSION/bin/ruby; then \
(apt-get update -y \
&& apt-get install -y -q gcp-ruby-$REQUESTED_RUBY_VERSION) \
|| (cd /rbenv/plugins/ruby-build \
&& git pull \
&& rbenv install -s $REQUESTED_RUBY_VERSION) \
&& rbenv global $REQUESTED_RUBY_VERSION \
&& gem install -q --no-rdoc --no-ri bundler --version $BUNDLER_VERSION \
&& apt-get clean \
&& rm -f /var/lib/apt/lists/*_*; \
fi
ENV RBENV_VERSION=${{REQUESTED_RUBY_VERSION:-$RBENV_VERSION}}
# Copy the application files.
COPY . /app/
# Install required gems if Gemfile.lock is present.
RUN if test -f Gemfile.lock; then \
bundle install --deployment --without="development test" \
&& rbenv rehash; \
fi
# Temporary. Will be moved to base image later.
ENV RACK_ENV=production \
RAILS_ENV=production \
RAILS_SERVE_STATIC_FILES=true
# Run asset pipeline if we're in a Rails app.
RUN if test -d app/assets -a -f config/application.rb; then \
bundle exec rake assets:precompile || true; \
fi
# BUG: Reset entrypoint to override base image.
ENTRYPOINT []
# Start application on port $PORT.
CMD {entrypoint}

View File

@@ -0,0 +1,5 @@
.dockerignore
Dockerfile
.git
.hg
.svn

View File

@@ -0,0 +1,462 @@
#!/usr/bin/python
# Copyright 2016 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import unittest
from gae_ext_runtime import testutil
RUNTIME_DEF_ROOT = os.path.dirname(os.path.dirname(__file__))
DOCKERFILE_TEXT = '''\
# This Dockerfile for a Ruby application was generated by gcloud.
# The base Dockerfile installs:
# * A number of packages needed by the Ruby runtime and by gems
# commonly used in Ruby web apps (such as libsqlite3)
# * A recent version of NodeJS
# * A recent version of the standard Ruby runtime to use by default
# * The bundler gem
FROM gcr.io/google-appengine/ruby:{base_image_tag}
# If your application requires a specific ruby version (compatible with rbenv),
# set it here. Leave blank to use the currently recommended default.
ARG REQUESTED_RUBY_VERSION="{ruby_version}"
# Install any requested ruby if not already preinstalled by the base image.
# Tries installing a prebuilt package first, then falls back to a source build.
RUN if test -n "$REQUESTED_RUBY_VERSION" -a \\
! -x /rbenv/versions/$REQUESTED_RUBY_VERSION/bin/ruby; then \\
(apt-get update -y \\
&& apt-get install -y -q gcp-ruby-$REQUESTED_RUBY_VERSION) \\
|| (cd /rbenv/plugins/ruby-build \\
&& git pull \\
&& rbenv install -s $REQUESTED_RUBY_VERSION) \\
&& rbenv global $REQUESTED_RUBY_VERSION \\
&& gem install -q --no-rdoc --no-ri bundler --version $BUNDLER_VERSION \\
&& apt-get clean \\
&& rm -f /var/lib/apt/lists/*_*; \\
fi
ENV RBENV_VERSION=${{REQUESTED_RUBY_VERSION:-$RBENV_VERSION}}
# Copy the application files.
COPY . /app/
# Install required gems if Gemfile.lock is present.
RUN if test -f Gemfile.lock; then \\
bundle install --deployment --without="development test" \\
&& rbenv rehash; \\
fi
# Temporary. Will be moved to base image later.
ENV RACK_ENV=production \\
RAILS_ENV=production \\
RAILS_SERVE_STATIC_FILES=true
# Run asset pipeline if we're in a Rails app.
RUN if test -d app/assets -a -f config/application.rb; then \\
bundle exec rake assets:precompile || true; \\
fi
# BUG: Reset entrypoint to override base image.
ENTRYPOINT []
# Start application on port $PORT.
CMD {entrypoint}
'''
class RuntimeTestCase(testutil.TestBase):
"""Tests for the Ruby external runtime fingerprinter."""
def file_contents(self, filename):
"""Reads the contents of the file from the tempdir.
Args:
filename: (str) filename to be joined with tempdir prefix.
Returns:
File contents.
"""
with open(self.full_path(filename)) as f:
return f.read()
def stub_response(self, response):
"""Stubs the console response from the user.
Args:
response: (str) stubbed response.
Returns:
A function to reset the stubbed functions to their original
implementations.
"""
can_prompt = self.exec_env.CanPrompt
prompt_response = self.exec_env.PromptResponse
def unstub():
self.exec_env.CanPrompt = can_prompt
self.exec_env.PromptResponse = prompt_response
self.exec_env.CanPrompt = lambda: True
self.exec_env.PromptResponse = lambda prompt: response
return unstub
def setUp(self):
self.runtime_def_root = RUNTIME_DEF_ROOT
super(RuntimeTestCase, self).setUp()
def test_generate_without_ruby_files(self):
self.write_file('index.html', 'index')
self.generate_configs()
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
self.assertFalse(os.path.exists(self.full_path('Dockerfile')))
self.assertFalse(os.path.exists(self.full_path('.dockerignore')))
def test_generate_without_ruby_files_no_write(self):
"""Tests generate_config_data does nothing if no ruby files."""
self.write_file('index.html', 'index')
self.assertIsNone(self.generate_config_data())
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
def test_generate_with_ruby_files(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
self.generate_configs()
unstub()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: ruby\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('entrypoint: bundle exec rackup -p $PORT -E deployment\n',
app_yaml)
self.assertFalse(os.path.exists(self.full_path('Dockerfile')))
self.assertFalse(os.path.exists(self.full_path('.dockerignore')))
def test_generate_with_ruby_files_no_write(self):
"""Tests generate_config_data with basic Ruby files.
Tests that app.yaml is written with correct contents given entrypoint
response, and that Dockerfile and .dockerignore not written to disk.
"""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
cfg_files = self.generate_config_data()
unstub()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: ruby\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('entrypoint: bundle exec rackup -p $PORT -E deployment\n',
app_yaml)
self.assertNotIn('Dockerfile', [f.filename for f in cfg_files])
self.assertNotIn('.dockerignore', [f.filename for f in cfg_files])
def test_generate_with_deploy(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
self.write_file('.ruby-version', 'rbx-3.9')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
self.generate_configs(deploy=True)
unstub()
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(
dockerfile,
DOCKERFILE_TEXT.format(
ruby_version='rbx-3.9',
entrypoint='bundle exec rackup -p $PORT -E deployment'))
dockerignore = self.file_contents('.dockerignore')
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_deploy_no_write(self):
"""Tests generate_config_data with deploy=True.
Tests that .dockerignore and Dockerfile contents are correct
based on contents of app.
"""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
self.write_file('.ruby-version', 'rbx-3.9')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
cfg_files = self.generate_config_data(deploy=True)
unstub()
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
DOCKERFILE_TEXT.format(
ruby_version='rbx-3.9',
entrypoint='bundle exec rackup -p $PORT -E deployment'))
self.assertIn('.dockerignore', [f.filename for f in cfg_files])
dockerignore = [f.contents for f in cfg_files if
f.filename == '.dockerignore'][0]
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_custom(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
self.generate_configs(custom=True)
unstub()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: custom\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('entrypoint: bundle exec rackup -p $PORT -E deployment\n',
app_yaml)
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(
dockerfile,
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec rackup -p $PORT -E deployment'))
dockerignore = self.file_contents('.dockerignore')
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_custom_no_write(self):
"""Tests generate_config_data with custom=True.
Tests that app.yaml is written with correct parameters and
Dockerfile, .dockerignore contents are correctly returned by method.
"""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
unstub = self.stub_response('bundle exec rackup -p $PORT -E deployment')
cfg_files = self.generate_config_data(custom=True)
unstub()
app_yaml = self.file_contents('app.yaml')
self.assertIn('runtime: custom\n', app_yaml)
self.assertIn('env: flex\n', app_yaml)
self.assertIn('entrypoint: bundle exec rackup -p $PORT -E deployment\n',
app_yaml)
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec rackup -p $PORT -E deployment'))
self.assertIn('.dockerignore', [f.filename for f in cfg_files])
dockerignore = [f.contents for f in cfg_files if
f.filename == '.dockerignore'][0]
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_existing_appinfo(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
appinfo = testutil.AppInfoFake(
entrypoint='bundle exec ruby index.rb $PORT',
runtime='ruby',
vm=True)
self.generate_configs(appinfo=appinfo, deploy=True)
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(
dockerfile,
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec ruby index.rb $PORT'))
dockerignore = self.file_contents('.dockerignore')
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_existing_appinfo_no_write(self):
"""Tests generate_config_data with passed appinfo."""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
appinfo = testutil.AppInfoFake(
entrypoint='bundle exec ruby index.rb $PORT',
runtime='ruby',
vm=True)
cfg_files = self.generate_config_data(appinfo=appinfo, deploy=True)
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec ruby index.rb $PORT'))
self.assertIn('.dockerignore', [f.filename for f in cfg_files])
dockerignore = [f.contents for f in cfg_files if
f.filename == '.dockerignore'][0]
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_ruby_version(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
self.write_file('.ruby-version', '2.3.1\n')
appinfo = testutil.AppInfoFake(
entrypoint='bundle exec ruby index.rb $PORT',
runtime='ruby',
vm=True)
self.generate_configs(appinfo=appinfo, deploy=True)
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(
dockerfile,
DOCKERFILE_TEXT.format(
ruby_version='2.3.1',
entrypoint='bundle exec ruby index.rb $PORT'))
dockerignore = self.file_contents('.dockerignore')
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_ruby_version_no_write(self):
"""Tests generate_config_data with .ruby-version file."""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
self.write_file('config.ru', 'run Index.app')
self.write_file('.ruby-version', '2.3.1\n')
appinfo = testutil.AppInfoFake(
entrypoint='bundle exec ruby index.rb $PORT',
runtime='ruby',
vm=True)
cfg_files = self.generate_config_data(appinfo=appinfo, deploy=True)
self.assertFalse(os.path.exists(self.full_path('app.yaml')))
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
DOCKERFILE_TEXT.format(
ruby_version='2.3.1',
entrypoint='bundle exec ruby index.rb $PORT'))
self.assertIn('.dockerignore', [f.filename for f in cfg_files])
dockerignore = [f.contents for f in cfg_files if
f.filename == '.dockerignore'][0]
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_prompt(self):
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
unstub = self.stub_response('bundle exec ruby index.rb $PORT')
self.generate_configs(deploy=True)
unstub()
dockerfile = self.file_contents('Dockerfile')
self.assertEqual(
dockerfile,
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec ruby index.rb $PORT'))
dockerignore = self.file_contents('.dockerignore')
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
def test_generate_with_prompt_no_write(self):
"""Tests generate_config_data with entrypoint given by prompt."""
self.write_file('index.rb', 'class Index; end')
self.write_file('Gemfile', 'source "https://rubygems.org"')
unstub = self.stub_response('bundle exec ruby index.rb $PORT')
cfg_files = self.generate_config_data(deploy=True)
unstub()
self.assert_genfile_exists_with_contents(
cfg_files,
'Dockerfile',
DOCKERFILE_TEXT.format(
ruby_version='',
entrypoint='bundle exec ruby index.rb $PORT'))
self.assertIn('.dockerignore', [f.filename for f in cfg_files])
dockerignore = [f.contents for f in cfg_files if
f.filename == '.dockerignore'][0]
self.assertIn('.dockerignore\n', dockerignore)
self.assertIn('Dockerfile\n', dockerignore)
self.assertIn('.git\n', dockerignore)
self.assertIn('.hg\n', dockerignore)
self.assertIn('.svn\n', dockerignore)
if __name__ == '__main__':
unittest.main()