Docker support updates: Local data stores and more languages

Today we're releasing some exciting improvements to the Heroku beta Docker support announced 3 months ago: Automatic configuration of local containers running data stores (like Postgres, Redis and MongoDB) and support for many more languages including images for Node.js, Ruby, Go, Java, Scala, Clojure and Gradle. This helps you use local containers to run, test and deploy Heroku apps that have complex service dependencies with minimal setup and configuration.

Heroku Docker support brings to your machine the Linux containerization technology that we have operated for many years. These local Docker containers make on-boarding, development and testing of apps simpler and faster, and with Docker Compose, you can quickly spin up apps consisting of multiple containers and services. With the new tools announced today, you can use Docker to test, run and deploy Heroku apps faster and more reliably. We can’t wait to see what you build.

How to

Continue reading “Docker support updates: Local data stores and more languages”

Heroku Button for Private Repos

Last year, we launched Heroku Button to make it simple for developers to deploy open source code to new Heroku apps. Open source contributors can add Heroku Buttons to GitHub READMEs, tutorials and blog posts and make their projects instantly deployable to Heroku, as apps fully provisioned with add-ons and other required configurations. Two months ago we introduced Elements where more than 1700 public Heroku Buttons are profiled alongside add-ons and top buildpacks.

Today, we're happy to announce Heroku Buttons for projects maintained by your team in private GitHub repos. This new feature uses Heroku's GitHub integration to securely deploy code referenced by buttons on private repos.

Buttons for private repos makes it simple to maintain private frameworks and quickstart template apps. Private repo buttons are great for:

Heroku Review Apps Beta

Today we’re announcing a feature that is going to change the way teams test and evaluate code changes. Continuous delivery works best when all team members — designers and testers included — can efficiently visualize and review the result of proposed changes. With Review Apps enabled, Heroku will spin up temporary test apps for every pull request that’s opened on GitHub, complete with fresh add-ons and other config required to make the app run. Instead of relying only on code reviews and unit tests run by CI, teams can use Review Apps to immediately try out and debug code branches in a production-like environment on Heroku. Review apps speed up team decision-making so that you can deliver better apps faster, and with greater confidence.

The Review Apps feature builds on the GitHub Integration announced in February and combines two things in this world that are good and righteous:

Introducing ‘heroku docker:release’: Build & Deploy Heroku Apps with Docker

When Heroku launched the Cedar container stack 4 years ago, we became one of the first companies to use Linux Containers (LXC) to create a secure, performant and scalable cloud platform. Heroku has been a leader in the containerization movement, and we’ve spent years hardening, honing and evolving our runtime container stack. This means that developers can git push apps written in their favorite language and Heroku will build containers that are deployed to a production-quality environment. With this approach, developers are set free from managing operating systems, package updates, server infrastructure and their inevitably complex interactions.

Containers are essential to making this model work, as they create firm abstractions and boundaries between an application’s code and all the dependant pieces necessary to make them run. And the benefits of containers for deploying and running apps are familiar to most developers using Heroku; freedom from having to manage down-stack components,

Continue reading “Introducing ‘heroku docker:release’: Build & Deploy Heroku Apps with Docker”

Ship Code Faster: Announcing GitHub Integration GA

Today we’re announcing the general availability of GitHub integration for Heroku. When enabled, GitHub pushes are deployed immediately to linked Heroku apps. This is a big step forward for people working on apps with source managed on GitHub and deployed to Heroku. The integration has been in beta in Heroku Dashboard for a while, and we’ve seen great adoption and positive feedback.

When a GitHub repo is connected to a Heroku app, you can either manually deploy branches from the repo, or you can configure a particular branch to auto-deploy to the app on every GitHub push. With auto-deploys enabled, you no longer have to maintain a separate heroku Git remote: Simply push to GitHub and changes are automatically deployed on Heroku.

Configuring GitHub integration

To connect a Heroku app with a GitHub repo, go to the app’s “Deploy” tab on Heroku Dashboard and select the GitHub pane. If you haven’t connected your Heroku and GitHub accounts, you will be prompted to complete the GitHub OAuth flow. Heroku needs access to help you select repos and to be able to register webhooks triggered when you push to GitHub. Once connected, you can select which repo associated with your GitHub account to link to the Heroku app.

gh-integration-screenshot

With app and repo connected, you can either manually deploy a specific branch, or select a Git branch that will be auto-deployed whenever it’s pushed to on GitHub.

For auto-deploys, you can optionally configure Heroku to wait for continuous integration (like Travis CI) to pass on GitHub. With that option enabled, Heroku will only auto-deploy after all GitHub statuses for that commit have succeeded.

Any builds created by the GitHub integration can be tracked in the app’s “Activity” tab and build output for running builds is streamed in Dashboard.

Continue reading “Ship Code Faster: Announcing GitHub Integration GA”

Update Git clients on Windows and OS X

Last week, a security fix was released for Git. The fix patches a bug in the Git client that is exploitable on operating systems with case insensitive file systems such as Windows and OS X.

Heroku has updated the Git installer that we ship with Toolbelt for Windows. We have also removed an old Git version from the OS X installer (it was not generally used).

In addition, we’ve added a Git version warning in Toolbelt that will prompt you to update Git if you’re using a vulnerable version on Windows (shown here) or OS X:

$ heroku apps
WARNING: Your version of git is 1.9.3. Which has serious security vulnerabilities.                                               
More information here: https://blog.heroku.com/archives/2014/12/23/update_your_git_clients_on_windows_and_os_x

Heroku Toolbelt ships msysGit for Windows and users should update to 1.9.5, available from the msysGit website. OS X users should update their system Git using, for example, the OS X installers or using Homebrew.

Details of the exploit are available on the Git Blame blog and from the Git mailing list announcement.

Continue reading “Update Git clients on Windows and OS X”

HTTP Git now Generally Available

Today we’re happy to announce that the HTTP Git beta is over and that HTTP Git is fully ready for production.

The beta was launched less than a month ago and we are already handling thousands of HTTP Git builds per day. In addition, HTTP Git powers the Dropbox Sync beta, making sure that Dropbox folders and Heroku repos are up-to-date.

Over the past month, we have seen great adoption from partners, and Travis CI is using HTTP Git as the default git strategy for Heroku deployments.

We encountered few issues during the beta, and we’re confident that HTTP Git is the best Git implementation for most Heroku users. For that reason, we’re making HTTP Git the default setup when repos are configured by Heroku tooling. We will keep SSH Git as an option and SSH Git still works for all repos on Heroku. You can use SSH Git and HTTP Git (and Dropbox Sync) interchangeably as needed by you and your collaborators. Read on for details.

With HTTP Git now the default, heroku create, heroku git:remote and heroku git:clone commands all configure your local environment to use HTTP Git. If, for any reason, you want to use SSH Git with a particular app, simply pass a --ssh-git flag to these commands, e.g.:

$ heroku create --ssh-git

If you want to always use SSH Git with Heroku on a particular machine, you can add the following global config:

git config --global url.
Continue reading "HTTP Git now Generally Available"

Announcing Beta Dropbox Sync

Helping teams to collaborate on creating, shipping and operating great apps is a core Heroku value. People collaborating on Heroku apps are not all alike: Some spend all day in the terminal, others prefer using Heroku from a browser. That’s why we’ve built both a powerful CLI and a great Dashboard.

Today, we’re adding beta support for Dropbox Sync to complement Git-based deployments. By adding Dropbox as a way to sync changes, we’re making it easier for more users on diverse teams to contribute to apps built on Heroku.

Git is a powerful tool for software developers to collaborate on building great apps and software. We added Git-based deployments 6 years ago, to plug Heroku straight into that collaboration flow and to make deploying to production as simple as sharing changes with your collaborators.

With Dropbox Sync, we’re inviting more people to participate in the process of creating great apps and we’re giving existing users more choice and flexibility in how code is deployed to Heroku. Read on for details.

How to Deploy with Dropbox

Getting set up with Dropbox deploys is incredibly simple and can be done from the comfort of a browser window. You can enable Dropbox Sync for both new apps and for existing apps that have existing releases created with Heroku Button or with Git.

Setup is done on the Code tab of an app and requires granting Heroku access to your Dropbox account. Note that Heroku will NOT have access to all the files and folders in your Dropbox account: Heroku’s access is limited to a special /Heroku subfolder.

dropbox-unlinked-app

Once setup is complete, a sub-folder will be created for the app in the /Apps/Heroku folder (the name of your /Apps folder may be localized). This is where source code for the app lives. If you enable Dropbox for an existing app that already has releases, Heroku will place the contents of the app’s Git repo into the Dropbox folder and you can start hacking right away.

Whenever you want to deploy your local changes to Heroku, simply enter a commit message and hit the “Deploy” button in Dashboard. Use the message to summarize in a few words what you have changed since the last deploy. Heroku will now fetch the latest version of your source code from Dropbox and kick off a build. You can immediately follow build progress and you can track progress and debug build errors from the “Activity” tab of the app.

Collaboration

Working together to create and operate apps is a core value of Heroku and one of the original motivations for building Git-based deployments. When designing and building Dropbox Sync, we worked hard to integrate the new feature into how source code is managed on the platform.

If you’re part of a development team where some use Git and others want to use Dropbox, fear not. Changes shipped with Dropbox are available to Git collaborators with a simple git pull and git push updates are automatically synced to connected Dropbox folders. If an app has multiple users deploying with Dropbox, the latest changes are synced to all connected Dropbox folders whenever a new app release is created. See Dev Center for details on how collaboration and conflict resolution works.

Git and Dropbox can be combined even if it’s just you working on an app. For example, you might do your day-to-day development on a laptop, pushing changes with Git. Later, if you need to make changes at a time and place where all you have is your iPad, no problem: simply open your code in a Dropbox-enabled editor, fix the problem, hit the “Deploy” button in Dashboard and you’re done.

Looking ahead

We’re excited about this beta and we’re very keen on getting your feedback. If you try out Dropbox Sync and encounter any problems or have suggestions or ideas, don’t hesitate to get in touch on dropbox-feedback@heroku.com.

We’re also really excited about the prospect of opening up Heroku to more kinds of developers. We’re working hard to ship new ways to get code onto Heroku: Both low-level APIs like the Build API and higher level services, like Dropbox Sync and Heroku Button built on those APIs.

Have ideas for other ways we should support moving source code to Heroku in addition to Git and Dropbox? Let us know or spike out your own prototype using the Build API.

Announcing HTTP Git Beta

Of the many Platform-as-a-Service innovations Heroku has contributed in its seven year existence, perhaps the most iconic is git push heroku master. Today we’re announcing a significant upgrade to Heroku’s Git implementation: Beta support for Git’s HTTP transport.

HTTP Git has some notable advantages over traditional SSH Git. Instead of relying on port 22 (often blocked by firewalls) HTTP Git runs on port 443, the same port used for secure web requests.

Also, HTTP Git uses a simpler authentication model than SSH Git, and is easier to set up. Many new users struggle with the tooling and configuration required to configure git-push over SSH, especially on Windows. HTTP Git uses Heroku API tokens for authentication, and Heroku Toolbelt takes care of setup and configuration so that you’re not prompted for your password on each push. See the Authentication section on Dev Center for details on how auth is managed.

How to use HTTP Git

Heroku Toolbelt has been updated and can be used to configure Git remotes that use HTTP. While in beta, you get HTTP by passing the --http-git flag to the relevant heroku apps:create, heroku git:clone and heroku git:remote commands. To create a new app and have it be configured with a HTTP Git remote, run this:

$ heroku apps:create --http-git

To change an existing app from SSH to HTTP Git, simply run this command from the app’s directory on your machine:

$ heroku git:remote --http-git
Git remote heroku updated

Check out the Dev Center documentation for details on how set up HTTP Git for Heroku.

Git and Heroku

The idea that Git can be used to deploy code to production is profound: It takes a process that you use many times a day to sync source code with collaborators, and applies it to deployments. This means that you don’t have to break flow and use different and unfamiliar tools to deploy. The result is that you’ll tend to deploy smaller changes more frequently (as opposed to giant, infrequent deploys), and that makes deploys less risky and bugs easier to identify and fix.

And because deployments derive directly from your revision control system, it’s very simple to determine what code is running on production. Simply run git checkout <deployed-commit> and you can start to reproduce a production issue with your app locally.

We’re very happy with the HTTP Git experience we’re launching into beta today, and we feel it’s on par or better than SSH Git across all the operating systems that Heroku supports. Once HTTP Git is out of beta, we expect to make it the default Heroku Git mode.

If you have feedback or suggestions for HTTP Git on Heroku, then don’t hesitate to get in touch on http-git-feedback@heroku.com.

Cedar-14 now Generally Available

We’re excited to announce that the Cedar-14 – the new version of the Celedon Cedar stack – is ready for general availability and is now the default stack on Heroku. Cedar-14 is based on the latest Ubuntu LTS Linux version and comes with a modern set of libraries and system dependencies that will stay current and updated for a long time to come.

Since we announced the public beta of Cedar-14 three months ago, we have migrated most of the apps that we run on Heroku to Cedar-14 (yes, a lot of Heroku runs on the Heroku platform) and thousands of users have also moved apps or created new Cedar-14 apps. We have worked with these early adopters and with buildpack maintainers to weed out bugs and problems. Cedar-14 is now stable and ready for production apps.

Read on to learn how to migrate apps to Cedar-14 and for details on future stack updates on Heroku.

Updating apps to Cedar-14

Before migrating a Heroku app to Cedar-14, we recommend testing your app source code on Cedar-14, either by upgrading an existing staging app or by creating a new staging app on Cedar-14.

To migrate an existing app to Cedar-14, use the stack command:

$ heroku stack:set cedar-14 -a example-app
…
$ git commit -m "update to cedar-14" --allow-empty
…
$ git push heroku master
…

Note that running stack:set only tells Heroku that you want the next push built and deployed on Cedar-14. You have to push and force a rebuild for the change to take effect. See the full Migrating to Cedar-14 Dev Center article for additional details.

If, in spite of testing, your app doesn’t work correctly on Cedar-14, you can always roll back to revert the stack change:

$ heroku rollback

Note that this will cause the apps stack to be reset to cedar and you need to re-run stack:set cedar-14 to deploy to Cedar-14.

What to look out for

We have worked hard to make migrating to Cedar-14 as smooth as possible, but there are a few things to look out for.

Stack image libraries

When putting together Cedar-14, we have strived to make it backwards compatible with the old Cedar stack. To reduce stack surface area and the frequency of potentially disruptive stack updates, we have, however, removed some lesser used packages from Cedar-14. Apps may also encounter incompatibilities simply because most libraries on Cedar-14 have been updated to their most recent versions. A full declaration of Cedar and Cedar-14 packages is available on Dev Center.

Memory use

One of the most significant package differences between Cedar and Cedar-14 is the glibc versions available. Our testing has found that some programs consume more memory on Cedar-14. This change is related to how address space is managed across threads in newer glibc versions. The Cedar-14 migration guide has more details, but in general, we recommend that you monitor app memory use during the migration to Cedar-14 (using the new Dashboard Metrics for example) and that you check for R14 memory errors after the migration.

Buildpack support for Cedar-14

Buildpacks are the scripts that transform your source code into slugs that run on the Heroku platform. Buildpacks often pull in binary dependencies that are specific to a particular stack and for that reason, they generally have to be made stack-aware.

We have updated all of the default buildpacks maintained by Heroku and we have have worked with maintainers of popular third-party buildpacks to get as many of those as possible updated too. If you find that a third-party buildpack you’re using doesn’t work on Cedar-14, we recommend that you reach out to the maintainer or that you move to a Heroku-supported buildpack. If you’re a buildpack maintainer, please refer the Buildpack API article for details on how to support multiple stacks.

Stack future

Cedar-14 is now the default stack on Heroku and we recommend you use that stack for new apps. If you need to create Cedar apps for testing purposes, you can still do so by passing --stack cedar when creating apps.

Stacks cannot live forever – packages and distro versions are deprecated or stop receiving security updates. With Cedar-14 GA, we’re also announcing the deprecation of the classic Cedar runtime stack: A year from now, on November 4th, 2015, the classic Cedar stack will be retired and any apps that have not been migrated to Cedar-14 will stop running. You must migrate all apps to Cedar-14 before this date to prevent disruption in availability. In the coming year, we will keep the old Cedar stack image patched and updated.

We know that Heroku users want stacks that are stable and have modern packages and libraries, and that’s why we currently base stacks on Ubuntu Linux LTS releases. We think Canonical and the Ubuntu community strike a great balance between stability and frequent updates with biennial LTS releases, and we plan to track those releases when we release new Heroku stacks.

A well-defined stack lifecycle policy is critical to keeping apps on Heroku patched secure. For additional details on how Heroku thinks about stacks and how we keep stacks updated, please refer to our Stack Image Update Policy on Dev Center.

Try the new Uber API on Heroku

Uber logo

On Wednesday, Uber launched an API to let developers build new products and services that leverage the Uber ridesharing platform. Uber built a simple Python/Flask app that developers can use when exploring how the API works.

This is the sort of experimentation and innovation that we at Heroku want to enable, so we sent a pull request to add an app.json file to the repo and a Heroku Button to the readme.

To deploy the Uber sample on Heroku and experiment with the new Uber API, simply register on the Uber developer site and then click the button below:

Deploy on Heroku

Once the app has been set up on Heroku, you’ll have to go back to the Uber developer site and configure the redirect URI to the URI for your newly created Heroku app.

That’s the power of Heroku: Setting up and configuring apps is quick and easy, and once you’re set up, scaling to über-size is just as simple.

Cedar-14 Public Beta

At Heroku, we want to give our users access to the latest and greatest software stacks to base their apps on. That’s why we continuously update buildpacks to support new language and framework versions and let users experiment further using third-party buildpacks.

Sitting underneath slugs and buildpacks are stacks. Stacks are the operating system and system libraries required to make apps run. Today we’re releasing into public beta a new version of the Celedon Cedar stack: cedar-14.

cedar-14 is built on the latest LTS version of Ubuntu Linux and has recent versions of libraries and system dependencies that will receive maintenance and security updates for a long time to come.

Before making cedar-14 the default stack, Heroku is looking for feedback from our users. To help weed out bugs and problems, please try out the stack with existing apps and new source code.

Creating a cedar-14 app

Here’s how to create a new app running on cedar-14:

$ heroku create --stack cedar-14
...

Pushing and deploying to cedar-14 apps works exactly the same as deploying apps on the classic Cedar stack.

Updating an app to cedar-14

Before migrating an existing app to cedar-14, we recommend either first upgrading an existing staging app or testing your app source in a new cedar-14 staging app.

To migrate an existing app to cedar-14, use the stack command:

$ heroku stack:set cedar-14 -a example-app
…
$ git commit -m “update to cedar-14” --allow-empty
…
$ git push heroku master
...

Note that running stack:set only tells Heroku that you want the next push to be built and deployed on a different stack. You have to push and force a rebuild for the change to take effect. See the full Migrating to Cedar-14 Dev Center article for additional details.

If, despite of testing, your app doesn’t work correctly on cedar-14, you can always use rollback to revert the stack change:

$ heroku rollback

Note that this will cause the apps stack to be reset to cedar and you need to re-run stack:set cedar-14 to deploy to cedar-14.

Compatibility

Stack compatibility is important to us, but we are also trying to take advantage of this opportunity to remove some less-used parts of the current stack surface area. We have documented the differences between the classic Cedar stack and cedar-14 beta stack on Dev Center. In addition to missing packages, apps may also encounter incompatibilities simply because most libraries on cedar-14 have been updated to their most recent versions. Please get in touch on stack-feedback@heroku.com or open a support ticket if your app fails to build or run on cedar-14 for any reason.

Buildpacks

We have updated all of the default buildpacks that we maintain to work with cedar-14 and are keen to get user feedback. We have also updated certain much used non-default buildpacks such as nginx-buildpack and buildpack-pgbouncer.

If you maintain a 3rd buildpack, please refer to the Buildpack API article for details on how to support multiple stacks. We want to help buildpack maintainers with this update process, so please don’t hesitate to reach out to stack-feedback@heroku.com with questions or concerns.

We’re looking forward to helping our users move to cedar-14, the latest and greatest Heroku stack. Apps running on cedar-14 will stay updated, fast and secure for a long time to come. Get in touch on stack-feedback@heroku.com if you encounter problems or have suggestions.

Introducing Heroku Button

At Heroku, we want to make the process of deploying, running and updating code simple and easy. To that end, we’re launching the Heroku Button: a simple HTML or Markdown snippet that can be added to READMEs, blog posts and other places where code lives. Clicking a Heroku Button will take you through a guided process to configure and deploy an app running the source code referenced by the button.

The best way to understand the Heroku Button is to try one. Click the example button below to deploy a Node.js sample project to an app running on your Heroku account:

Deploy on Heroku

How it works

When you see a Heroku Button in a README, in documentation or in a blog post, then this is a piece of code that’s ready to deploy and run on Heroku.

When you click a Heroku Button, you access a setup flow in Dashboard. If you’re not logged in or don’t have an account, you’ll go through the login flow first. Heroku then determines what code you’re trying to deploy (using either the referer header or a parameter in the button URL). Heroku uses an app.json manifest in the code repo to figure out what add-ons, config and other deployment steps are required to make the code run. This is used to configure and deploy the app.

button-step-1

Once you have provided any required config and confirmed the setup, Heroku will start deploying the source to a new app on your account:

button-step-2

When setup is complete, you can open the deployed app in your browser or inspect it in Dashboard.

button-step-3

To make changes, simply clone from the Heroku git repo as you would any other app, modify to your liking and push the changes:

$ heroku clone -a my-new-app
…
$ vim index.js
$ git add .
$ git commit -m “update index.js”
$ git push heroku master
...

The Heroku Button replaces previous README-defined guides to clone code, create an app, provision add-ons, run setup scripts and git-push. All these steps and requirements can now be encoded in an app.json file and placed in a repo alongside a button that kicks off the setup with a single click.

Adding a Heroku Button

Adding a Heroku Button to a piece of code that you care about is simple. Currently, code hosted in public repos on GitHub is supported.

The first step is to add an app.json to the root of the repo. App.json is a manifest format for describing apps and specifying what their config requirements are. Heroku uses this file to figure out how code in a particular repo should be deployed on the platform. Here’s a sample app.json file:

{
  "name": "Node.js Sample",
  "description": "A barebones Node.js app using Express 4",
  "repository": "https://github.com/heroku/button-sample",
  "logo": "https://node-js-sample.herokuapp.com/node.svg",
  "keywords": ["node", "express", "static"]
}

With the app.json file in place, the next step is to create the actual button. The button can live on any web page you like, but a good start is probably the README file in the repo that source code you want people to deploy. Here’s an example Markdown snippet:

[![Deploy](https://www.herokucdn.com/deploy/button.png)](https://heroku.com/deploy)

Note that the snippet is repo-agnostic: It can be copy-pasted without modification and will work correctly if forked to a different repo. Heroku resolves the repo originating a button click by inspecting the referer header. HTML is also an option and you can also use Heroku Buttons outside repo READMEs if you add a query parameter to the button URL.

The Dev Center documentation has details on how to create and maintain app.json files and links to additional resources.

Looking ahead

We’re excited about the Heroku Button: It’s going to obviate reams of README on public repos describing how to configure apps to run on Heroku.

We have worked with several add-on providers to take advantage of buttons to make experimenting with their services on Heroku easier. Check out the CloudAMQP and MemCachier Dev Center articles and notice how sample apps in different languages can be deployed and tested with a simple click.

Twilio Logo

Our good friends at Twilio have added a deploy button to their Rapid Response Kit, a suite of communication tools that can be used for anything from emergency response to community organization. With the button in the README, interested developers can immediately deploy the Rapid Response Kit to Heroku and start hacking on it to make it do what they need.

We are also using Heroku Button for the tools and software that we ourselves support and maintain. Yesterday we released Starboard, a tool for managing onboarding and offboarding of employees. The README comes with a Heroku Button to make it easy for others to try out and build on our work.

SF72 Logo

And the Heroku Button is cropping up in other places too: Check out the City72 app, a preparedness platform built by the San Francisco Department of Emergency Management and IDEO. The app runs the sf72.org site to help San Franciscans prepare for emergencies and with the Heroku Button in place, it’s now very simple for other cities and communities to deploy their own preparedness sites on Heroku.

We’re looking for more Heroku Buttons, so if you maintain open source code and if you want to make deploying that code to Heroku super simple, you should consider adding a Heroku Button to your README. Check out the Heroku Button docs for more details.

And when your button is live, let us know via Twitter by using #HerokuButton. We'll be listening and will retweet some of our favorites.

Introducing programmatic builds on Heroku

Today, we are announcing an important addition to the Heroku Platform API: The /apps/:app/builds endpoint. This endpoint exposes the Heroku slug compilation process as a simple API primitive. You can use the endpoint to turn any publicly hosted source-tarball into a slug running on a Heroku app in seconds.

Here’s output from a Go program that invokes the new endpoint:

$ ./build -app limitless-fjord-5604 -archive https://github.com/heroku/node-js-sample/archive/master.tar.gz
.........
-----> Node.js app detected
-----> Requested node range:  0.10.x
-----> Resolved node version: 0.10.28
-----> Downloading and installing node
…
$ curl http://limitless-fjord-5604.herokuapp.com/
Hello World!

Here’s what is going on: an app name is passed to the script along with a URL to a tarball containing source code that we want to build and deploy to the app. For this example, we’re using a tarball generated by GitHub, containing a hello-world Node.js app.

The script takes these inputs and uses /apps/:app/builds to create a new build on Heroku. The Platform API responds with the Id of the new build. New builds starts out in pending state. The Go program starts polling for status of the build. In the background, the build service fetches the tarball, unpacks it, and feeds the source through the standard Heroku slug compiler.

Once the build has completed, the state changes from pending to either succeeded or failed. The Go program then fetches the build output using the result endpoint and prints it in the terminal. The build is automatically deployed to the app and we can cURL it to get a Hello World! response.

Uses for the builds endpoint

The builds endpoint exposes the full power of the Heroku slug compiler and release-infrastructure in a very simple API. It is designed for use in automated CI deployment flows. The builds endpoint complements the slug and release endpoints that we announced last year. Combining these endpoints, developers can create flows where source code is pushed to a repository (not on Heroku), built into a slug by invoking the /apps/:app/builds endpoint and then deployed to one or more apps on the platform using the slug and release endpoints.

The builds endpoint has been in private beta for a while and we already have a few cool uses. @atmos has built hubot-deploy, a chat-bot plugin that lets you deploy to Heroku from your favorite hubot-enabled chat room. And we have created a deploy plugin for hk.

What we’re releasing today is the first version of the builds endpoint. We have already gotten lots of great feedback and based on that, we are considering future additions such as:

  • Ability to POST tarballs directly (versus accepting link to downloadable tarball)
  • Adding streaming build output as the build progresses
  • Making releasing to app optional (currently, all successful builds are deployed immediately)
  • Support for fetching source using the Git protocol

To get started with the builds endpoint, check out the Dev Center tutorial, inspect the reference docs or have a look at the Go sample app.

We are always keen to hear user’s thoughts on what we build. For API feedback, get in touch at api-feedback@heroku.com or create a discussion topic on the Heroku Forums.

Heroku Platform API now GA

Today, we’re happy to announce General Availability of the Heroku Platform API. Heroku is a platform built by developers, for developers. As developers, we understand the utility of APIs and the power APIs give to speed up and script error-prone manual processes or to combine other services with Heroku into new and exciting products. With the Platform API, you now have a fully documented and supported way to instrument and automate Heroku.

Platform API

Designing and implementing this API has been an important process for Heroku internally: It has forced us rethink how different platform components are factored and how they should be exposed in a clean and coherent manner. We are already using the Platform API heavily for projects at Heroku, and the new API is superseding and replacing the spread of private and semi-public APIs that have made Heroku work so far.

Because platform functionality is now available in a single API that has good documentation and updated client libraries, Heroku can create better products and services, faster. By making the API public, we want to give Heroku customers, users, and partners the same opportunity.

Readers new to the Plaform API can jump straight to the quickstart to get started.

APIs are hard

Building and maintaining APIs for the long haul is hard. As an API evolves, documentation has to be kept in sync and client libraries for supported languages must be updated to take advantage of new features. To make those tasks easier for ourselves, Heroku publishes a single canonical JSON schema that is used to generated the artifacts necessary to consume the Platform API, including Reference Documentation and client libraries for Ruby, Go, Scala and Node.

UPDATE: Check out this follow-up blog post on the Heroku HTTP API Toolchain we have built to manage this process.

Another difficult aspect of API maintenance is managing and communicating changes. To address this, we have recently added stability attributes to the JSON schema for the Platform API. Stability is also surfaced in the reference doc on Dev Center. For any given endpoint, stability can be either prototype, development or production. The stability of an endpoint communicates how “done” the endpoint is. A prototype endpoint will likely see many changes or may not make it to production. A production endpoint, on the other hand, is done and will not see breaking changes unless the underlying feature is deprecated (which will be communicated with at least 12 months warning).

All API endpoints that expose core Heroku concepts (such as account, app and add-on) are of production stability. Heroku users and partners can incorporate these API endpoints into their workflows and services safe in the knowledge that the API will not change or go away without due warning.

Heroku also publishes all API changes to the Changelog and we recommend subscribing for updates.

Looking ahead

The past year has been a busy one for the Platform API. The beta launched last May, OAuth came in July and we have been adding endpoints and methods at a rapid clip since. To get an idea of what we’re currently working on, you can skim the reference docs and look for prototype and development endpoints.

To get started with the Platform API, check out the Getting Started Guide. For personal experiments and scripts used to manage apps on your account, we recommend using the API key found on your account page in Dashboard. You can also generate additional keys for experiments using the API. The Platform API also has beta support for OAuth, but we have not yet perfected the OAuth scopes nor how token access is scoped to endpoints exposed in the API.

We are also looking at combining the Platform API with the API that most Heroku partners currently use to integrate with Heroku: The Add-on provider API. Until we have completed this task, OAuth clients can have tokens issued for at most 100 Heroku accounts. If you are building an OAuth-enabled service on the Platform API and expect that it will get wide distribution, we ask that you contact us to discuss a partnership.

We are always keen to hear user’s thoughts on what we build. For API feedback, get in touch at api-feedback@heroku.com or create a discussion topic on the Heroku Forums.

Two-factor Auth in Public Beta

Today, we’re excited to announce public beta of two-factor authentication for Heroku accounts. With two-factor auth enabled, an authentication code is required whenever you log in. The code is delivered using an app on your smartphone, and access to your phone becomes a required factor (in addition to your password) to access Heroku. An attacker that has somehow discovered your password will not be able to log in using just your password.

Enabling two-factor auth

The easiest way to enable two-factor auth is using Dashboard. Go to your account page, click the “Enable two-factor authentication” button and follow the on-screen instructions.

Enable two-factor auth button

Download an authenticator app for your smartphone if you don’t already have one. We recommend Google Authenticator but alternatives like Authy work too.

Two-factor apps

Scan the barcode shown on the Dashboard page using the downloaded authentication app.

Bar code

Finally, enter the 6-digit code displayed on your smartphone to enable two-factor authentication.

Verify code

That’s it! Your account is now protected with two-factor auth.

In the coming months, we want to add support for sending authentication codes using SMS and we’ll expose two-factor auth support in the Platform API, so stay tuned.

At Heroku we care deeply about the security of apps and accounts on the platform. In the past month alone, we responded vigorously to the Heartbleed vulnerability, launched our Security Bug Bounty program and announced important security improvements for SSL endpoints. With two-factor auth enabled, your Heroku account will have an extra layer of security that stops attackers that have somehow discovered your password. Enable it now.

JSON Schema for the Heroku Platform API

Today we’re making an important piece of Platform API tooling available: A machine-readable JSON schema that describes what resources are available via the API, what their URLs are, how they are represented and what operations they support. The schema opens up many interesting use cases, both for users and for us at Heroku working on improving and extending the API. A few examples are:

  • Auto-creating client libraries for your favorite programming language
  • Generating up-to-date reference docs
  • Writing automatic acceptance and integration tests

We are already using the schema to maintain the API reference documentation on Dev Center and to generate several v3 client libraries:

Of these, Heroics is officially supported by Heroku and the other three are community projects created by Heroku engineers. If you find bugs or have suggestions for enhancements, please send a pull request. We’re interested in having even more client libraries and hope hackers in the Heroku community will start building libs for other languages, either auto-generated ones using JSON Schema or hand-written ones.

How to use the Schema

The API serves up its own JSON-formatted schema using HTTP:

$ curl https://api.heroku.com/schema -H "Accept: application/vnd.heroku+json; version=3"
{
  "description": "The platform API empowers developers to automate, extend and combine Heroku with other services.",
  "definitions": {
  …
  }
}

Let’s take a closer look at the schema definition for the app object (edited for brevity):

"app": {
  "description": "An app represents the program that you would like to deploy and run on Heroku.",
  ...
  "definitions": {
    "name": {
      "description": "unique name of app",
      "example": "example",
      "pattern": "^[a-z][a-z0-9-]{3,30}$",
      "type": [
        "string"
      ]
    }
  },
  "links": [
    {
      "description": "Create a new app.",
      "href": "/apps",
      "method": "POST",
      "rel": "create",
      "schema": {
        "properties": {
          "name": {
            "$ref": "#/definitions/app/definitions/name"
          }
        }
      },
      "title": "Create"
    }
  ],
  "properties": {
    "name": {
      "$ref": "#/definitions/app/definitions/name"
    }
  }
}

From the schema, we can surmise that the app object has a name property of type string. We even get a regex describing acceptable values for the name property.

The “links” element describes the methods available on the app object. The sample above only includes the POST method used to create a new app. The link object includes a schema property that shows that specifying a name property is optional when creating an app.

Feel free to explore the full schema to see what other information is available and check out the Heroics README and source code for ideas on how to use it.

JSON Schema

When deciding on a format to describe and document the API, we had the following requirements:

  • JSON-based, like the rest of the content served by the API
  • Clean and simple enough to allow human perusal
  • Expressive enough to facilitate generating documentation and client libraries

As much as possible, we also wanted the format to be based on open standards to allow re-use of existing code and tooling.

WSDL and its cousin WADL are probably the most well-known mechanisms for describing web services. They weren’t appropriate for Heroku’s API, however: WSDL is XML-based, complicated and rather verbose.

Instead, we settled on JSON Schema for the base format. Combined with the draft Validation and Hypertext extensions, we had the expressiveness required to meet our design objectives with a JSON format.

As a bonus, both Swagger and the Google API discovery service are built on JSON Schema and it’s a fairly well-established standard with lots of tooling and other resources already available.

Summary

Publishing a machine-readable schema for your API is good practice: It makes keeping docs up to date easier and it greatly simplifies maintenance of up-to-date client libraries. We think JSON Schema is a solid foundation for describing APIs and we will continue to invest in the format and it’s extensions and in relevant tooling that make using APIs easier.

We also hope the Heroku community will take advantage of the API schema to build and contribute to clients and tools in addition to the ones we release today. Our goal is to make the Platform API easy to consume from a wide variety of languages and frameworks.

If you come up with something cool and want to tell us about it then drop us a line on api-feedback@heroku.com. If you have questions about how to use the schema or comments on its design, please create a topic in the Platform API Forum.

Programmatically release code to Heroku

Currently in beta, the Heroku Platform API lets developers automate, extend and combine the Heroku platform with other services in a programmatic, self-service way. Today we are setting the capstone into the API by adding slug and release endpoints to the API beta.

These API endpoints are special. They expose a very core part of what Heroku does best: Quickly and safely releasing new versions of an app onto the Heroku platform.

Using the new slug and release endpoints, platform developers can build integrations and services that completely sidestep the traditional Heroku Git deployment flow. So instead of requiring git push heroku master to deploy, it’s now possible to do things like:

  • Write a script that builds an app in a local directory on your machine, tars it up, and deploys it directly to a Heroku app
  • Extend your favorite continuous integration server or service to deploy directly to Heroku if unit tests pass – no need to use Git or manage SSH keys
  • Move slugs around between Heroku apps, for example to promote a release on a staging app to a production app

Are you a CI service provider, source code repository provider, or just a plain old hacker interested in the opportunities this opens up? We want to work with you, get in touch with friis@heroku.com.

The slug and release endpoints complete the functionality that we plan to expose in the first version of the Heroku Platform API. The API is still in beta and we welcome questions and feedback in the Heroku API Forum.

Slugs and Releases

Platform API

“Slug” refers to one of two things:

  1. A tarballed bundle of source code, fetched dependencies and compiled or generated output of a build system, ready for execution on the Heroku runtime
  2. A record created in the Heroku database when you register and upload a slug tarball using the slug endpoint on the platform API

The slug API endpoint allows slug records to be created and read. It contains attributes including the URL of the slug tarball, the process types that the slug can run (mirroring the contents of the Procfile), and a version control identifier (typically the SHA from the Git commit of the source code used to create the slug)

Releases are versioned deployments of an app, containing a slug and a set of config vars. The release API endpoint allows releases to be created and read. Releases require a slug and can optionally contain a description.

Slugs can be shared amongst releases – by fetching the slug from an existing release on any given app, a new release can be created on any other app using the existing slug. This allows slugs to be quickly and efficiently promoted between environments (such as from staging to production), and the release endpoint can be used to programmatically create custom deployment workflows.

The rest of this post contains two examples that demonstrate the power of the new API endpoints. The first shows how to move slugs between apps, the second demonstrates how to create and deploy slugs from scratch, from your local machine.

Full documentation is available on Dev Center in these articles:

Copying slugs

There are several ways to get a valid slug that can be released to an app. The simplest method is to copy a slug generated by Heroku for a different app. Imagine that you have pushed code to a staging app, the code has been tested and you are now ready to release to production. Instead of pushing to the production app and waiting for the code to build again, you can simply copy the slug from the staging app to the production app.

First, list releases on the staging app to get the id of the slug to release:

$ curl -H "Accept: application/vnd.heroku+json; version=3" -n \
https://api.heroku.com/apps/example-app-staging/releases
...
"slug":{ "id":"ff40c84f-a538-4b65-a838-88fdd5245f4b" }

Now, create a new release on the production app using the slug from the staging app:

$ curl -X POST -H "Accept: application/vnd.heroku+json; version=3" -n \
-H "Content-Type: application/json" \
-d '{"slug": "ff40c84f-a538-4b65-a838-88fdd5245f4b"}' \
https://api.heroku.com/apps/example-app-production/releases

That’s it! The new code is now running on the example-app-production app.

Note that copying slugs between apps is already possible using the beta Pipelines plugin, and Heroku Fork uses a similar mechanism when forking an app. The slug and releases endpoints expose the primitives necessary for third-party API developers to build services offering similar functionality and much more.

Creating slugs from scratch

Slugs don’t have to be generated by Heroku: You can use the releases endpoint to deploy anything the platform recognizes as a valid slug. Let’s create a slug containing a simple Node.js app and the dependencies required to run it on Heroku. On your local machine, create a folder named app to hold the slug and fetch the Node.js runtime:

$ mkdir app 
$ cd app
$ curl http://nodejs.org/dist/v0.10.20/node-v0.10.20-linux-x64.tar.gz | tar xzv

Add the app code in a file named web.js:

// Load the http module to create an http server
var http = require('http');

// Configure HTTP server to respond with `Hello World` to all requests
var server = http.createServer(function (request, response) {
  response.writeHead(200, {"Content-Type": "text/plain"});
  response.end("Hello World\n");
});

var port = process.env.PORT;

// Listen on assigned port
server.listen(port);

// Put a friendly message on the terminal
console.log("Server listening on port " + port);

Tar up the slug:

$ cd ..
$ tar czfv slug.tgz ./app

The above process of creating a slug is a very simplified variant of what buildpacks do when normal slugs are created on Heroku: Make sure the source code is placed where it’s supposed to be and package any dependencies (the Node.js runtime in this case) into the slug.

The slug is now ready for release. This is a three-step process:

  1. Create a unique id and URL for the slug using the new slug endpoint
  2. Upload the slug
  3. Release the slug on the app

First, we register the new slug using the API. We have to keep track of the “put” url and the id that Heroku returns:

$ curl -X POST -H 'Content-Type: application/json' \
-H 'Accept: application/vnd.heroku+json; version=3' \
-d '{"process_types":{"web":"node-v0.10.20-linux-x64/bin/node web.js"}}' \
-n https://api.heroku.com/apps/example-app/slugs
...
  "blob":{
    "method": "put",
    "url": "https://s3-external-1.amazonaws.com/herokuslugs/heroku.com/..."
  },
  "id":"d969e0b3-9892-3113-7653-1aa1d1108bc3"

Notice that we pass a “process_types” parameter. This has content similar to what is included in the Procfile that is typically included in apps pushed to Heroku. Passing process_types is necessary because the Procfile is not parsed when slugs are launched in dynos. Instead, the Procfile is parsed by the build system and the contents are passed in when the slug is created as demonstrated above.

Use curl to upload the slug to the URL provided by Heroku:

$ curl -X PUT -H "Content-Type:" --data-binary @slug.tgz "https://s3-external-1.amazonaws.com/herokuslugs/heroku.com/…"

Finally, release the slug to the app example-app:

$ curl -X POST -H "Accept: application/vnd.heroku+json; version=3" \
-H "Content-Type: application/json" \
-d '{"slug":"d969e0b3-9892-3113-7653-1aa1d1108bc3"}' \
-n https://api.heroku.com/apps/example-app/releases

We can then check that the release was created:

$ heroku releases --app example-app
=== example-app Releases
v3  deploy  example@example.com  2013/10/08 16:09:54 (~ 1m ago)

To verify that the slug was deployed, run$ heroku open --app example-app. Please see the full example on Dev Center for additional details on how to create slugs. The Dev Center article has both the Node.js example above and examples demonstrating how to create simple slugs with Ruby and Go apps.

Summary

With the releases and slugs endpoints, the primitives that power Heroku features like Pipelines and Fork are now available to developers using the Platform API. This opens up a lot of exciting possibilities for partners and hackers to build and innovate on top of Heroku. We will be working with partners to build integrations as we take the API to general availability. Get in touch with friis@heroku.com if you are interested in participating.

The release and slug endpoints are in public beta with the rest of the Platform API. Once we’re confident the API is free of bugs, we will freeze the design and release it into GA. Until that time, we may introduce breaking changes to the API. All changes will be posted in the Heroku Changelog.

If you have any questions or feedback, please start a discussion in the Heroku API Forum.

OAuth for Platform API in Public Beta

In May, we launched the beta Heroku Platform API – making it possible to automate, extend and combine the Heroku platform with other services in a programmatic, self-service way. As of today, OAuth 2.0 support for the Platform API is available in public beta.

With OAuth support, developers building integrations and services that use the Heroku API can provide a much better experience to their users. Instead of requesting full access to user accounts, access requests can be scoped to just the information and control a service needs. Instead of using one API key for all third-party services, users can check and revoke authorizations on a case-by-case basis. And users can manage all of their third-party authorizations within their Heroku dashboard.

API Developers

If you are building a service that uses the Platform API, you should implement OAuth. First, register a client from the account page on Dashboard. You can then incorporate OAuth into your app using OmniAuth, the Heroku Bouncer middleware or another tool of your choice. The Heroku OAuth article has additional details, resources and links to sample apps.

For Heroku Users

With the Platform API, developers can now build awesome services that integrate with Heroku – for example, an iPhone app to monitor apps running on Heroku, or a CI service that can push changes to your apps so your workflow is smoother and more automated. However, these services need access to some or all of your Heroku account to work. OAuth gives you a safe mechanism to control this access. When a service uses OAuth to request access to your account, you will be redirected to id.heroku.com where you can see who is requesting access and the scope of the access requested. Here are the scopes we have implemented so far:

  • global: Full access to account and to control all apps and resources. Equivalent to API key (but revocable).
  • identity: Read-only access to account-info.
  • read and write: Access to read and write info on apps and other resources, except configuration variables. This scope lets you grant access to your apps without necessarily revealing runtime secrets such as database connection strings.
  • read-protected and write-protected: Same as above, but including access to config vars.

Note that the read, write, read-protected and write-protected scopes do not grant access to account identity details such as email address.

Before granting access to a 3rd party service, make sure that you trust that service to access your Heroku account in a way you feel comfortable with. You can see what external services are authorized on the account page on Dashboard or using the authorizations CLI command from the OAuth CLI plugin. You can revoke authorizations at any time using either the dashboard or Heroku CLI.

OAuth gives 3rd parties services safe, granular and revocable access to the power of the Heroku Platform API. We can’t wait to see what new apps and services get built with these technologies.

If you have questions, suggestions or want to show us what you have created, then drop us a line at api-feedback@heroku.com.

Heroku Labs: Managing App Deployment with Pipelines

Editor's Note: Features added through Heroku Labs are experimental and may change or be removed without notice.

heroku fork lets you create unique, running instances of existing applications in a single command, making it fast and simple to set up homogenous development, staging and production environments. But have you ever wished you could deploy directly from staging to a production app after testing and validation?

Heroku pipelines, now an experimental feature available in Heroku Labs, lets you define the relationship between apps and easily promote a slug from one app to another. On Heroku, a slug is a bundle of your source, fetched dependencies, the language runtime, and compiled/generated output of the build system, ready for execution.

Pipelines will copy the slug from the upstream app to the downstream app. Time to deploy is significantly faster as the slug is already compiled – pipelines simply copies and moves it to the downstream app. Pipelines also reduces the risk of accidentally pushing the wrong commit or deploying to the wrong environment, and makes it easier to manage your workflow so you can develop and test new features quickly, verify expected behavior, and release safely to end users. Here’s how to try it.

Set Up Pipelines

To use pipelines, you must first enable it in Heroku Labs:

$ heroku labs:enable pipelines

Then install the CLI plugin:

$ heroku plugins:install git://github.com/heroku/heroku-pipeline.git

Defining a Workflow

If you don’t already have your application environments set up, use heroku fork to quickly spin up unique, homogeneous application environments from existing apps. You can then use these instances for various stages of your workflow.

For this example, let’s assume that myapp-production is the name of your existing app, and you want to create a fork called myapp-staging:

$ heroku fork -a myapp-production myapp-staging

This will clone the source app’s precompiled slug and config-vars, re-provision the source app’s add-ons, copy over Heroku Postgres data (if present), and scale web processes to one dyno in the new myapp-staging environment. For more on heroku fork, its behaviors and limitations, read the blog post.

Next, we use pipeline to map the relationship between the staging and production apps. In pipeline vocabulary, a production app is "downstream" from a staging app. Given a dev —> staging —> production pipeline, staging would be downstream of dev, and production downstream of staging.

For simplicity’s sake, let’s set up a basic staging-to-production workflow between myapp-staging and myapp-production. Add the production app as the downstream environment:

$ heroku pipeline:add -a myapp-staging myapp-production

Confirm by calling heroku pipeline:

$ heroku pipeline -a myapp-staging
Pipeline: myapp-staging ---> myapp-production

Diff Between Application Environments

Diff is a powerful tool for seeing changes between code, here extended to entire application environments. Diff against your downstream app:

$ heroku pipeline:diff
Comparing myapp-staging to myapp-production ...done, myapp-staging ahead by 1 commit:
73ab415  2013-07-07  A super important fix  (Shanley)

Promote Changes

Once you’ve defined a pipeline, you can deploy from one app to a downstream app with one command:

$ heroku pipeline:promote
Promoting myapp-staging to myapp-production...done, v2

This will copy the upstream app’s currently running slug to the downstream app as a new release. In this example, application changes made in staging will now be live and running in your production app, ready to serve end users.

Pipelines only manage the application slug. Config vars, Add-ons and other environmental dependencies are not considered part of a pipeline and must be managed independently. Similarly, pipeline doesn't update the downstream app's repository, so a heroku git:clone of a downstream app will not retrieve the latest code base.

If you're adding application code that interacts with an Add-on, but that Add-on is not yet present in a downstream app, you'll want to provision the Add-on there before promoting application changes to it. Promoted changes by mistake, or need to roll back? Revert a promotion on the downstream app with heroku rollback to deploy a previous, known version of your code.

Feedback?

Heroku Labs lets us get experimental features out early and often so you can try them and tell us what you like and how we can improve. Check out our current Labs features, and email us at labs@heroku.com.