drupal development. drupal support. drupal training.

Follow SystemSeed on:

Do it with Drupal

The Seed Feed

At SystemSeed, we use Jenkins to trigger automated platform builds directly from commits to our git code repositories. A lot of our code is hosted on github, although due to the nature of our work, some of our client repositories are private. Our Jenkins build scripts are heavily based on Mig5's article Zero-touch Drupal deployment with Jenkins, Aegir, Git, Fabric and Drush which enables drush make files to be fetched and build directly from the web. However, one difference is that our repositories are private, and therefore inaccessible to drush make directly. Yesterday, I found a way to overcome this by using Github's API directly with drush make.

Actually, to say I found this out yesterday is not quite true. We have been doing this for some months, though for some reason our old method suddenly stopped working, so I found myself having to revist the solution.

Drush Make allows you to build a remote drush make file by simply supplying the URL to the make file, like so

https://raw.github.com/systemseed/private_platform/7.x-3.x/private_platf...

This of course assumes that the private_platform repository on githib is publically accessible - which in our case it is not. In the past, this could be worked around by invcluding an OAuth token in the url, like so.

https://<TOKEN>@raw.github.com/systemseed/private_platform/7.x-3.x/priva...

In the above, <TOKEN> should be replaced with an OAuth token that you generate on github.

Unfortunately, this method no longer works. But, it is still possible to access these private repositiories directly - it just take a little more work. It all boils down to using the new github version 3 api in order to authenticate and fetch the private data. Here are the steps:

Generate an OAuth token

In order to access your private data through the github api you need to generate an OAuth token. There are lots of articles about how to do this, but this is a simple one that I like that shows how you can generate a github OAuth token easily from the command line.

Look up the file using the github api

Once you have your token, you can use this to start querying the api. What you need to do here is find out the api url that you can use to access your drush make build file. In our case, the file we are looking for is called private_platform.make, which is located at the root of the 7.x-3.x branch of the private_platform repository. To access this using hte github api, you would need to use the Blobs API. However, in order to use the Blobs API you first need to know the SHA hash of the file that you are looking for, which can be found out using the Repo Contents API. For example, 

curl -i https://api.github.com/repos/systemseed/private_platform/contents/privat...

This will return details of the private_platform.make file in the 7.x-3.x branch, including the blob url which will be showin the the git_url attribute of the response. The URL will look simething like https://api.github.com/repos/systemseed/private_platform/git/blobs/31997.... This URL can now be used to fetch the raw content of that file.

Configure curl to request the data in raw format

Now that you know the api url from where the file content can be fetched, you need to tell the api to return just the raw file data. By default, the github api will return the data as a base64 encoded string, wrapped in some json. Since we want to pass this file directly to drush make, this is no good for us. Luckily, github lets us use media types in order to control the format of the response. By sending the Accept header application/vnd.github.v3.raw along with our request, github will return the raw file content for us.

curl -i -H "Accept: application/vnd.github.v3.raw"  https://api.github.com/repos/systemseed/private_platform/\
git/blobs/31997c60b3c2e85e789dcf1a13b934c3a3fdb730?access_token=<TOKEN>

with Drush make, we can't actually pass request headers through like this. But, we can set them in a ~/.curlrc file, which is a file that lets us set default options for curl to use. To do this add the following to your ~/.curlrc file (if the file doesn not already exist, you can just create it).

--header "Accept: application/vnd.github.v3.raw"

Adapt your drush make scripts

After all that, we now have a URL that we can pass directly to drush make so that it is able to fetch the content of a drush make file that is hosted in a private repository on github!

drush make "https://api.github.com/repos/systemseed/private_platform/\
git/blobs/31997c60b3c2e85e789dcf1a13b934c3a3fdb730?access_token=<TOKEN>" /path/to/build

After years of developing all types of web solutions, Tom made the strategic decision to focus his efforts into making Drupal a better platform. In 2010, he led the successful exit of his KirkDesigns through a joint venture with Web at Ease. That event formed SystemSeed.