cloudship

Cloudship is a Clojure toolkit to explore and manipulate your salesforce instances. It’s designed to work from the repl against different orgs at he same time and supports

• Auth Data from different Sources

  • Username + Password or Session in Configuration

  • sfdx

  • OAuth via web-server flow

• Extensions to Configuration

  • get Username + Password from Keepass

  • connect via CipherCloud

  • your own extension

• Convenient methods taking and returning Clojure Datastructures via

  • SOAP-Api

  • Bulk-Api

  • Meta-Api (describe and read for now)

  • csv-Files (cast with the describe-data of an existing org)

Quickstart

Include cloudship in your project or clone this project and start a repl using lein repl (you will need Leiningen for this).

(require '[cloudship.data :as data])
(data/q :mycon.auth:web "Account" [:Id] {:limit 10})

The second call should open a browser and starts an OAuth flow. You can then login into your Developer Org and when the flow is finished, cloudship will use the session the query the Ids of the first 10 Accounts on your org and return them.

Also the session is cached for one hour and every subsequent call using the key :mycon.auth:web will use the cached session.

You are up and running to explore the api further.

How it works

Every function that talks to a Salesforce Org (cloudship.data and cloudship.metadata) takes a cloudship as first argument (usally a keyword as in the quickstart).

While working with your data and metadata when you have an established connection is pretty straight forward, (and you can probably stop reading here if you only have one dev-org to toy around) the configuration and connection resolving is a bit sophisticated.

This is because cloudship is developed to easily access many orgs/sandboxes from the repl fast and at the same time. For example if :myprod is configured to connect to your production org, :myprod:mydev automaticly resolves to the sandbox mydev of your org and :myprod.v:40 connects to your production using api-version 40.

So here is a explanation how the resolval works.

Getting a Connection

When cloudship resolves a keyword or a basic config map to a connection there are four steps:

• parse keyword into basic config map (when starting with a keyword)

• expand into a full config map

• run an auth mechanism

• cache this information (including the resulting session) for fast reuse

Start with a Keyword

A keyword that describes a cloudship has the following syntax:

:org-name[:sandbox-name][.flag|.flag:opt]*

where the org-name can be choosen freely, sandbox-name is the name of the sandbox you want to connect.

Given this keyword, cloudship will do the following:

1. check if there is a cached CloudshipClient Record for this keyword

2. if not, build a property map from the keyword by

   1. parsing the keyword into a initial prop map (with :org-name :sandbox :flags and :cache-name (the full keyword) as keys)

   2. continue with the [From Property Map]

To see this in action run

(cloudship.connection.props.keyword/kw->props :your:keyword)`

Expand Property Map

Given a property map, cloudship will try to create an CloudshipClient Record by

1. deep-merges all found cloudship.edn in resource-folder, %user-home/.cloudship, current folder (in this order) to a big config map.

2. deep-merges to a final prop map

   1. all entries (except :orgs) of config map (1.)

   2. all entries in [:orgs :org-name] of config map

   3. all entries in [:orgs :org-name :sandboxes "sandbox-name"] of config-map

   4. the initial prop map

3. add or coerce api-version (currently default is 53.0)

4. build base-url from :sandbox, :instance and :my-domain

   • providing nothing will result in login.salesforce.com

   • only sandbox will result in test.salesforce.com

   • only my-domain will result in <my-domain>.my.salesforce.com

   • my-domain, sandbox needs the instance (e.g. "86") as this cannot be resolved and will result in <my-domain>--<sandbox>.cs<instance>.my.salesforce.com

5. Resolving Flags

6. add https:// to url if missing

7. add proxy if there is a system default proxy

8. add .<sandbox> to username if target is sandbox and it’s missing

To see 1. and 2. in action run

(cloudship.connection.props.load/find-and-merge-props "props from kw->props"})

To see the final property map you can run

(cloudship.connection.props.core/->props :your:keyword:or:map)

Run Auth

With the resulting property map, cloudship tries to auth against your org. Currently there are three auth methods (you can configure with :auth-method in your config).

• :soap - uses the login method of the soap client (default)

  • needs :username and :password (supports :security-token)

  • or just an existing :session

• :sfdx - gets session by calling sfdx:force:org:display -u ${:username or :org}]

• :web - uses the web server OAuth flow

  • you can provide :consumer-key, :consumer-secret, callback-port, callback-timeout

  • if you connect to an own app, make sure to set the callback-url to https://localhost.

Resolve Examples

Given the cloudship.edn

{:api-version "51.0"
 :orgs {:org1 {:username "[email protected]"
               :password "very-secret1!"
               :sandboxes {"new" {:api-version "52.0"}}}
        :my-dev {:kppath ["mydev" "login"]
                 :kpdb "dir/to/db.kdbx"
                 :my-domain "cloudship"}}}

the keywords will result in the following property maps

(->props :org1) =>
    {:api-version "51.0",
     :full :org1,
     :org "org1",
     :password "very-secret1!",
     :url "https://login.salesforce.com",
     :username "[email protected]"}

(->props :org1:new) =>
; api-version is overwritten in config
; username and url are adjusted for the sandbox
    {:api-version "52.0",
     :base-username "[email protected]",
     :full :org1:new,
     :org "org1",
     :password "very-secret1!",
     :sandbox "new",
     :url "https://test.salesforce.com",
     :username "[email protected]"}

(->props :org1:other) =>
; nothing is overwritten, but username/url are adjusted for the sandbox
    {:api-version "51.0",
     :base-username "[email protected]",
     :full :org1:other,
     :org "org1",
     :password "very-secret1!",
     :sandbox "other",
     :url "https://test.salesforce.com",
     :username "[email protected]"}

(->props :org1.v:39) =>
; version is adjusted by the flag
    {:api-version "39.0",
     :full :org1.v:39,
     :org "org1",
     :password "very-secret1!",
     :resolved-flags [{:flag-name "v", :opt "39"}],
     :url "https://login.salesforce.com",
     :username "[email protected]"}

Resolving Flags

Flags are instructions to modify the connection properties before trying to create the CloudshipClient Record.

A Flag can either be a simple String or a Map with {:flag-name string? …​}. The Multimethod cloudship.connection.props.flags/resolve-flag is called on the flag and needs to return a function Property Map → Property Map.

All Flags are resolved and applied in order until there are no more left.

The following flags are included:

• v - Version flag that sets the version to it’s opt e.g. v:39.

• kp - Keepass, reads username and password from a keepass file.

  • needs :kpdb (path to keepath db) and :kppath (vector of path in keepath db).

  • :kppass can be set, otherwise will be prompted.

• auth short to set the auth-method e.g. auth:sfdx

• sfdx short for auth:sfdx

  • needs sfdx executable in path

  • uses :username or :org as $usernameOrAlias

• web short for auth:web

• cc - CipherCloud, changes the url to a CipherCloud url.

  • needs :cc-domain and :cc-alias.

  • you probably won’t need this.

Advanced Topics

Missing API Methods

• There a methods the SOAP/Metadata API provides that are not available in the cloudship API. If you need one of them you can directly call the methods you need against the PartnerConnection/MetadataConnection which are included in the (info []) call (path: [$client-type :base :connection]).

Known Issues

Java SDK Bulk API

• As the Bulk-API is build with the csv content type, inner queries are not supported

• Also because of this, it’s impossible to know wether SELECT Account.Phone FROM Contact returns Account.Phone as nil because Phone is nil or because there is no Account attached to the Contact. In this case {:type "Contact" :Account nil} will be returned. To avoid this, make sure to query a related field that can’t be nil (like Id).

License

Copyright © 2019 Albrecht Schmidt

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.