2011-10-27
R Job Reference
Neptune has support for R, and deploys in a non-distributed environment over a single machine.
All the possible parameters that can be used with R jobs are:
:type (required) - The type of job to run. For R jobs, this should be set to :r.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:code (required) - The location in the underlying storage system of the compiled code to run. A :compile job can be used to compile R source.
:output (required) - The location in the underlying storage system that the result of the job (the standard out and standard error) should be saved to.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
Go Job Reference
Neptune has support for Go, and deploys in a non-distributed environment over a single machine.
All the possible parameters that can be used with Go jobs are:
:type (required) - The type of job to run. For Go jobs, this should be set to :go.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:code (required) - The location in the underlying storage system of the compiled code to run. A :compile job can be used to compile Go source.
:output (required) - The location in the underlying storage system that the result of the job (the standard out and standard error) should be saved to.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
Output Job Reference
To get either the results of a compute job or the code used for a compute job, you use an output job. The results or code are located in the storage backend that you're using. This could be AppDB (whatever database that AppScale is using), or a remote database. Remote databases currently supported are Amazon S3, Google Storage, or Eucalyptus Walrus (as all three are API-compliant).
The parameters that can be used in an input job are:
:type (required) - The type of job to run. For output jobs, this should be set to :output.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:output (required) - The location on the remote storage system where the file should be copied over from. This location should begin with a slash in the usual POSIX fashion.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
Input Job Reference
Before running a compute job, you first have to put some input (usually the code to compute with) in the storage backend that you're using. This could be AppDB (whatever database that AppScale is using), or a remote database. Remote databases currently supported are Amazon S3, Google Storage, or Eucalyptus Walrus (as all three are API-compliant).
The parameters that can be used in an input job are:
:type (required) - The type of job to run. For input jobs, this should be set to :input.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:local (required) - The location on the local file system of the file to copy over. This should definitely point to a file that (1) exists and (2) is readable by the current user.
:remote (required) - The location on the remote storage system where the file should be copied over to. This location should begin with a slash in the usual POSIX fashion.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
UPC Job Reference
Neptune has support for UPC 2.12.1, and deploys in a distributed environment using the MPI backend.
All the possible parameters that can be used with UPC jobs are:
:type (required) - The type of job to run. For UPC jobs, this should be set to :upc.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:nodes_to_use (required) - The number of machines to run the UPC job over.
:procs_to_use (optional) - The total number of processes to run (not the number of processes per node). Must be at least as large as :nodes_to_use.
:code (required) - The location in the underlying storage system of the code to run.
:output (required) - The location in the underlying storage system that the result of the job (the standard out and standard error) should be saved to.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
X10 Job Reference
Neptune has support for X10 2.1.0, and deploys in a distributed environment using the MPI backend.
All the possible parameters that can be used with X10 jobs are:
:type (required) - The type of job to run. For X10 jobs, this should be set to :x10.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:nodes_to_use (required) - The number of machines to run the X10 job over.
:procs_to_use (optional) - The total number of processes to run (not the number of processes per node). Must be at least as large as :nodes_to_use.
:code (required) - The location in the underlying storage system of the code to run.
:output (required) - The location in the underlying storage system that the result of the job (the standard out and standard error) should be saved to.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
2011-10-27
MPI Job Reference
Let's say you want to run a MPI job, but don't know how. Well here are all the possible parameters that can be used with MPI jobs:
:type (required) - The type of job to run. For MPI jobs, this should be set to :mpi.
:keyname (optional) - The keyname associated with the AppScale cloud that this job should be run over. Defaults to 'appscale' if not provided.
:nodes_to_use (required) - The number of machines to run the MPI job over.
:procs_to_use (optional) - The total number of processes to run (not the number of processes per node). Must be at least as large as :nodes_to_use.
:code (required) - The location in the underlying storage system of the code to run.
:output (required) - The location in the underlying storage system that the result of the job (the standard out and standard error) should be saved to.
:storage (optional) - The storage backend to use. Defaults to 'appdb' (whatever database AppScale is running), and can also be 's3', 'gstorage', or 'walrus', for Amazon S3, Google Storage, or Eucalyptus Walrus, respectively.
:EC2_ACCESS_KEY (optional) - The EC2 access key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:EC2_SECRET_KEY (optional) - The EC2 secret key to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
:S3_URL (optional) - The S3 URL to use when using S3, Google Storage, or Eucalyptus Walrus as the storage backend.
