Test iOS Apps with UI Automation: Bug Hunting Made Easy now in beta
When we think of the concept of Waza (技) or "art and technique," it's easy to get caught up in the idea of individual mastery. It's true that works of art are often created by those with great skill, but acquiring that skill is neither solitary nor static. Generations of masters contribute to a canon and it is in that spirit that we built the Heroku platform and the Waza event. This year's Waza was no exception.
On February 28th, more than 900 attendees participated in Waza including Ruby founder Yukihiro "Matz" Matsumoto, Django co-creator Jacob Kaplan-Moss and Codeacademy’s Linda Liukas. True to form, we offered you a platform for experimentation and you surprised us with your contributions.
From your origami creations, to your Arduino hacks, to the technical conversations over craft beer — you taught us that the definition of software development is ever-evolving. Thank you for allowing us to help you change lives and push boundaries. We will continue our commitment to growing the platform for you and look forward to collaborating with you in the future.
Modern C++ Programming with Test-Driven Development now in beta
If you’re building a customer-facing web app or mobile backend, performance is a critical part of user experience. Fast is a feature, and affects everything from conversion rates to your site’s search ranking.
The first step in performance tuning is getting visibility into the app’s web performance in production. For this, we turn to the app’s logs.
Logs as data
There are many ways to collect metrics, the most common being direct instrumentation into the app. New Relic, Librato, and Hosted Graphite are cloud services that use this approach, and there are numerous roll-your-own options like StatsD and Metrics.
Another approach is to send metrics to the logs. Beginning with the idea that logs are event streams, we can use logs for a holistic view of the app: your code, and the infrastructure that surrounds it (such as the Heroku router). Mark McGranaghan’s Logs as Data and Ryan Daigle’s 5 Steps to Better Application Logging offer an overview of the logs-as-data approach.
Put simply, logs as data means writing semi-structured data to your app's logs via STDOUT. Then the logs can be consumed by one or more services to do dashboards, long-term trending, and threshold alerting.
The benefits of logs-as-data over direct instrumentation include:
- No additional library dependencies for your app
- No CPU cost to your dyno by in-app instrumentation
- Introspection capability by reading the logs directly
- Metrics backends can be swapped out without changes to app code
- Possible to split the log stream and send it to multiple backends, for different views and alerting on the same data
Introducing log2viz, a public experiment
log2viz is an open-source demonstration of the logs-as-data concept for Heroku apps. Log in and select one of your apps to see a live-updating dashboard of its web activity.
For example, here’s a screenshot of log2viz running against the Rubygems Bundler API (written and maintained by Terence Lee, André Arko, and Larry Marburger, and running on Heroku):
log2viz gets all of its data from the Heroku log stream — the same data you see when running
heroku logs --tail at the command line. It requires no changes to your app code and works for apps written in any language and web framework, demonstrating some of the benefits of logs as data.
Also introducing: log-runtime-metrics
To enable this for your app (and see memory stats in log2viz), type the following:
$ heroku labs:enable log-runtime-metrics -a myapp $ heroku restart
This inserts data into your logs like this:
heroku[web.1]: measure=load_avg_5m val=0.0 heroku[web.1]: measure=memory_total val=209.64 units=MB
log2viz reads these stats and displays average and max memory use across your dynos. (Like all Labs features, this is experimental and the format may change in the future.)
Looking under the hood
log2viz is open source. Let’s look at the code:
- It uses OAuth via
heroku-bouncerto connect to your Heroku account and make API calls to access information about your apps.
- It reads from your app's log stream. This is the same API endpoint that the Heroku CLI connects to when you run
heroku logs --tail.
- It's written in Ruby and uses the Sinatra streaming API (an EventMachine implementation of Server-sent Events) to handle both the incoming log data and outgoing live updates to the user's browser.
- It uses the Heroku API when you connect to the page to fetch the app's name and number of web dynos. It takes a hint for concurrency per dyno based on the
- All of the metrics are derived from 60 seconds of data, stored in-memory in the user’s browser. (Storing historical data would enable trend analysis, but we chose to leave that out of scope for this experiment.)
- log2viz only subscribes to the app’s logs while the page is open in your browser. Closing the window also closes the log connection.
log2viz isn't the only way to take advantage of your log stream for visibility on Heroku today. Here are a few add-ons which consume your app's logs.
Loggly offers a web console that lets you search your log history, and graph event types over time. For example, let’s search for status=404, logged by the Heroku router whenever your app serves a page not found:
Papertrail offers search archival and history, and can also alert on events when they pass a certain threshold. Here’s how you can set up an email alert every time your app experiences more than 10 H12 errors in a 60 second period. Search for the router log line:
Click “Save Search,” then:
You can also use non-add-on cloud services, as shown in thoughtbot's writeup on using Splunk Storm with Heroku.
Visibility is a vast and challenging problem space. The logs-as-data approach is still young, and log2viz is just an experiment to get us started. We look forward to your feedback on log2viz, log visibility via add-ons, and your own experiments on performance visibility.
Registrations are now open for RubyLearning’s popular Ruby programming course. This is an intensive, online course for beginners that helps you get started with Ruby programming.
Download ‘Advice for Ruby Beginners’ as a .zip file.
Here is what Demetris Demetriou, a participant who just graduated, has to say – “When I joined this course I was sceptical about how useful this course would be for me instead of reading material and watching videos on YouTube and thus saving money. After the course started I realised how valuable this course was. In the past I had read many Ruby books over and over, but never got into really getting practical with it and never had confidence in it. Lots of theory but couldn’t use it. I feel that the exercises in this course and the support, monitoring from our mentor Victor, made the huge difference that all books in the past didn’t. It wasn’t about reading lots of books, but simply few things and get practical and understand them well. I feel I learnt a lot and I’m coming back for more to rubylearning.org Thanks a lot Victor and Satish and all the other Rubyists who gave us today’s Ruby.”
According to http://www.ruby-lang.org/en/ – “Ruby is a dynamic, open source programming language with a focus on simplicity and productivity. Ruby’s elegant syntax is natural to read and easy to write.”
Yukihiro Matsumoto, the creator of Ruby, in an interview says –
I believe people want to express themselves when they program. They don’t want to fight with the language. Programming languages must feel natural to programmers. I tried to make people enjoy programming and concentrate on the fun and creative part of programming when they use Ruby.
What Will I Learn?
In the Ruby programming course, you will learn the essential features of Ruby that you will end up using every day. You will also be introduced to Git, GitHub, HTTP concepts, RubyGems, Rack and Heroku.
Depending on participation levels, we throw a Ruby coding challenge in the mix, right for the level we are at. We have been known to give out a prize or two for the ‘best’ solution.
Who’s It For?
A beginner with some knowledge of programming.
You can read what past participants have to say about the course.
Satish Talim, Michael Kohl, Satoshi Asakawa, Victor Goff III and others from the RubyLearning team.
The course starts on Saturday, 6th Apr. 2013 and runs for seven weeks.
RubyLearning’s IRC Channel
Most of the mentors and students hang out at RubyLearning’s IRC (irc.freenode.net) channel (#rubylearning.org) for both technical and non-technical discussions. Everyone benefits with the active discussions on Ruby with the mentors.
How do I register and pay the course fees?
- The course is based on the The Ultimate Guide to Ruby Programming eBook. This book is priced at US$ 9.95
- You can pay either by Paypal or send cash via Western Union Money Transfer or by bank transfer (if you are in India). The fees collected helps RubyLearning maintain the site, this Ruby course, the Ruby eBook, and provide quality content to you.
At the end of this course you should have all the knowledge to explore the wonderful world of Ruby on your own.
Here are some details on how the course works:
Once the course starts, you can login and start with the lessons any day and time and post your queries in the forum under the relevant lessons. Someone shall always be there to answer them. Just to set the expectations correctly, there is no real-time ‘webcasting’.
- The Mentors shall give you URL’s of pages and sometimes some extra notes; you need to read through. Read the pre-class reading material at a convenient time of your choice – the dates mentioned are just for your guideline. While reading, please make a note of all your doubts, queries, questions, clarifications, comments about the lesson and after you have completed all the pages, post these on the forum under the relevant lesson. There could be some questions that relate to something that has not been mentioned or discussed by the mentors thus far; you could post the same too. Please remember that with every post, do mention the operating system of your computer.
- The mentor shall highlight the important points that you need to remember for that day’s session.
- There could be exercises every day. Please do them.
- Participate in the forum for asking and answering questions or starting discussions. Share knowledge, and exchange ideas among yourselves during the course period. Participants are strongly encouraged to post technical questions, interesting articles, tools, sample programs or anything that is relevant to the class / lesson. Please do not post a simple "Thank you" note or "Hello" message to the forum. Please be aware that these messages are considered noises by people subscribed to the forum.
Outline of Work Expectations:
- Most of the days, you will have exercises to solve. These are there to help you assimilate whatever you have learned till then.
- Some days may have some extra assignments / food for thought articles / programs
- Above all, do take part in the relevant forums. Past participants will confirm that they learned the best by active participation.
Some Commonly Asked Questions
- Qs. Is there any specific time when I need to be online?
Ans. No. You need not be online at a specific time of the day.
- Qs. Is it important for me to take part in the course forums?
Ans. YES. You must Participate in the forum(s) for asking and answering questions or starting discussions. Share knowledge, and exchange ideas among yourselves (participants) during the course period. Participants are strongly encouraged to post technical questions, interesting articles, tools, sample programs or anything that is relevant to the class / lesson. Past participants will confirm that they learned the best by active participation.
- Qs. How much time do I need to spend online for a course, in a day?
Ans. This will vary from person to person. All depends upon your comfort level and the amount of time you want to spend on a particular lesson or task.
- Qs. Is there any specific set time for feedback (e.g., any mentor responds to me within 24 hours?)
Ans. Normally somebody should answer your query / question within 24 hours.
- Qs. What happens if nobody answers my questions / queries?
Ans. Normally, that will not happen. In case you feel that your question / query is not answered, then please post the same in the thread – “Any UnAnswered Questions / Queries”.
- Qs. What happens to the class (or forums) after a course is over? Can you keep it open for a few more days so that students can complete and discuss too?
Ans. The course and its forum is open for a month after the last day of the course.
Remember, the idea is to have fun learning Ruby.
(Powered by LaunchBit)
On February 16th, we published a blog post outlining five specific and immediate actions we would take to improve our Rails customers' experience with Heroku. We want to provide you with an update on where these things stand. As a reminder, here’s what we committed to do:
- Improve our documentation so that it accurately reflects how our service works across both Bamboo and Cedar stacks
- Remove incorrect and confusing metrics reported by Heroku or partner services like New Relic
- Add metrics that let customers determine queuing impact on application response times
- Provide additional tools that developers can use to augment our latency and queuing metrics
- Work to better support concurrent-request Rails apps on Cedar
We have resolved the first two items:
Improving Documentation: We’ve updated our Dev Center docs and website to more accurately describe how routing occurs on Heroku (e.g., HTTP Routing on Bamboo and How Routing Works on the Bamboo Stack).
Removing Incorrect Metrics: We’ve worked with New Relic to release an updated version of their monitoring tools, which we documented in Better Queuing Metrics With Updated New Relic Add-on.
We are working on the remaining three items in partnership with our early beta users:
- Adding New Metrics: We are improving the data available through our logs in two ways. First, we’re adding a transaction ID to every request log. Second, we are working on application middleware to provide additional metrics in customer logs.
- Providing Additional Measurement Tools: We have created a data visualization tool that analyzes log data in real-time and helps observe an app’s performance across a few key Heroku-specific metrics. This tool is currently being beta tested.
- Improving Cedar Support: Unicorn is now the recommended Rails app server. We are currently beta testing larger dynos to support additional unicorn processes.
Fulfilling these commitments is Heroku’s number one priority. We have dedicated engineering and product teams focused on improving the performance of Rails apps on Heroku.
We’ll detail the availability of our improvements in future blog posts. If you would like to beta test any of these new features, please drop us a note and let us know which features you are interested in testing.
Functional Programming Patterns in Scala and Clojure now in beta
The value of a network is proportional to the square of the number of users connected to the system, according to Metcalfe’s law. Jacob Kaplan-Moss, co-creator of Django, highlights this as a value in creating communities or as he puts it, “ecosystems”. In his talk at Waza last week on building ecosystems, he went on to highlight three key principles of creating ecosystems:
- APIs to support extensibility
- Conservatism as a value
- Empowering the community
Whether building a platform or a framework, these key principles ring true. Check out his talk or read more on creating ecosystems below:
APIs to support extensibility
You don’t choose to become an ecosystem, but you can choose to do the groundwork. By defining APIs you enable others to build on the foundation you lay. Django core member Russ Keith-McGee summed this up quite clearly on the Django mailing list – "I’m not sure if this is appropriate, but it should be possible. If its not possible we should build the APIs to support it." This sentiment is at the core of building ecosystems.
Django succeeded at this with its apps API, and has a full directory in Django Packages to help others discover what has already been built. At Heroku we’re seeing the same phenomon with our buildpacks API resulting in numerous third-party buildpacks. By building APIs and empowering users, you solve more problems.
Conservatism as a value
There’s great value in the “move fast and break stuff” concept. However, creating explicit APIs and contracts that are stable is critical for an ecosystem to thrive longer term. For Django as a framework this means being slow moving and reliable when it comes to modifying any APIs in the core framework. For Heroku as a platform it means a dedication to erosion resistance. Jacob cleanly highlights this by pointing out that “Interoperabilty only works when the common parts don’t change.”
Empowering the community
Ecosystems create ways for the community to engage and contribute. With an ecosystem there’s even additional ways in which users can contribute and those contributions don’t have to be code. When a project first starts, it’s creator tests it and consumes it acting as developer and QA. As it gains popularity a project may attract new developers, then more users, who submit their own bug reports and issues. These newcomers contribute documentation patches, triage tickets, and recruit others to help with the project. Where once a single developer committed quietly, now a thriving ecosystem of contributors and consumers exists.
This change from product to ecosystem doesn’t happen overnight, it must be nurtured and nudged. Kaplan-Moss, a self-proclaimed “documentation nut” has done just that by cultivating a better understanding of Django through documentation and carefully working with patch and issue submitters. Jacob explains that if you document your process, you can increase transparency and understanding of the way things work. Django does this by clearly documenting how users can contribute back to Django; at Heroku we’ve done this by documenting our methodology on how apps should be built.
Building a great product isn’t enough, as Jacob points out: you’ve got to create a vibrant community and build an ecosystem. If it wasn’t for Django’s community there wouldn’t be over 3500 Python packages for Django. If it wasn’t for our community, you wouldn’t be able to deploy Go, common lisp, or GeoDjango to Heroku with just one command.
The ActiveModel::Serializers gem can help you build JSON APIs through serializer objects. This provides a dedicated place to fully customize the JSON output.
Welcome to this week’s roundup of Ruby news, articles, videos, and more, cobbled together from my e-mail newsletter, Ruby Weekly. Sorry these roundups have been missing for a couple of months, I’ve been focusing very heavily on the e-mail newsletters which are continuing to grow like crazy! 🙂 I hope to get back into blogging more soon.
Matz on Ruby 2.0
Matz spoke about Ruby 2.0 (‘the happiest release ever’) for 30 minutes at the Heroku Waza event a week ago and the video is already available to watch. He stresses that “Ruby 1.8 will die soon” and encourages everyone to upgrade.
Dynamic Method Definitions
Aaron ‘tenderlove’ Patterson says that “depending on your app, using define_method is faster on boot, consumes less memory, and probably doesn’t signigicantly impact performance” compared to eval-based techniques. (And he has the numbers to prove it.)
Steel City Ruby Conference 2013 CFP Now Open
Steel City Ruby takes places in Pittsburgh, PA on August 16-17 and the CFP is now open if you want to submit a talk. The Burlington Ruby Conference has a CFP open too, as does RubyConf India.
Inspecting Rails 4 using Ruby 2.0 and TracePoint
Matt Aimonetti shows off a practical use for Ruby 2.0’s TracePoint execution tracing functionality.
Visualizing Memory Leaks in Ruby 1.9
Conrad Irwin on some clever work to extend ObjectSpace with a new find_references method to perform better analysis on object and memory usage on Ruby 1.9.
Parsing TOML in Ruby with Parslet
Recently, GitHub founder Tom Preston-Werner created an interesting INI-influenced ‘TOML’ format. In this series of posts, Nathan Witmer looks at what’s involved in building a parser for TOML using the Parslet PEG parser construction library.
Introducing Ress: A System for Building Mobile Optimized Rails Apps
Matthew Robertson introduces his new system for building mobile-optimized Rails applications using semantic, media query-based device detection and server side component optimization.
Ruby 2.0 Walkthrough: The Best Bits
Some slides from my yet-to-be-released ‘Ruby 2.0 Walkthrough’ that quickly skim through what I consider to be the ‘best bits’ (and not just the headline features).
Rails + Ember.js
Watching and Listening
Sinatra in SIX Lines: How to Do Crazy Stuff with Ruby
A talk by Konstantin Haase at Ruby Australia.
Libraries and Code
Phusion Passenger 4.0 Release Candidate 4
Leading Rack-based app deployment tool Passenger gets yet another step closer to the 4.0 release.
time-lord: A Human DSL for Time Expressions
A gem that gives you more human like expressions for time and space math. Get fun like
identity_cache: Opt-in Read-through ActiveRecord Caching, From Shopify
IdentityCache lets you specify how you want to cache your model objects, at the model level, and adds a number of convenience methods for accessing those objects through the cache. Uses Memcached as the backend cache store.
neg 1.1.0: A Small PEG Parser Library
“One could say it’s a small brother of Parslet.”
Web Application Developer for Big Nerd Ranch
Seeking smart, kind folks who want to make the world a little better through developing, training and writing about cutting-edge code.
JS / Ruby Developer at ReplayPoker (Full-Time, Remote)
Looking for a challenge? Our company is looking for a top-notch junior to mid level developer to join our small team and make a big difference!
Last but not least..
RTanque: A Robot Programming Game for Rubyists
Players program the ‘brain’ of a tank and then send their tank into battle with other bots. Based upon the Java project ‘Robocode.’
Matz, the creator of Ruby, spoke at Waza for the 20th anniversary of the language and the release of Ruby 2.0. If you weren't in the sold out crowd, not to worry. Information should flow free and experiences should be shared; in line with those concepts you can watch Matz's talk right here, then read about what's new in this version of Ruby and how to run it on Heroku.
With slides available on speakerdeck
Iterating quickly means happier developers and happier customers. We optimize our platform to allow you to develop, stage, and deploy faster. Not only does this make running software easier, but it makes trying new technologies like Ruby 2.0 as simple as spinning up a new app. During Matz's talk at Waza, he mentioned that, while 1.9.3 is popular now, it took years after 1.8.7 was released to gain traction. With the release of Ruby 2.0 Matz hopes to reduce upgrading barriers and allow developers to iterate quicker using newer, faster and better tools.
Ruby 2.0 was written to be backwards compatible and it works with Rails 3.2.13 out of the box. If your Ruby apps are running using 1.8.7, you should upgrade. Ruby 1.8.7 is approaching End of Life (EOL) in three months on June 2013. EOL for Ruby 1.8.7 means no security or bug patches will be provided by the maintainers. Not upgrading means you're potentially opening up your application and your users to vulnerabilities. Don't wait till the final hour, upgrade now to be confident and secure.
Ruby 2.0 has a faster garbage collector and is Copy on Write friendly. Copy on Write or COW is an optimization that can reduce the memory footprint of a Ruby process when it is copied. Instead of allocating duplicate memory when a process is forked, COW allows multiple processes to share the same memory until one of the processes needs to modify a piece of information. Depending on the program, this optimization can dramatically reduce the amount of memory used to run multiple processes. Most Ruby programs are memory bound, so reducing your memory footprint with Ruby 2.0 may allow you to run more processes in fewer dynos.
If you’re not already running a concurrent backend consider trying the Unicorn web server.
In addition to running faster than 1.9.3, and having a smaller footprint, Ruby 2.0 has a number of new features added to the language including:
- Keyword arguments
- Kernel#require optimization which makes Rails startup very fast
- Copy on write
- Dtrace support
__dir__to get the current directory of
- UTF-8 is now the default encoding
- Much, much more
The list of new features is more than we can cover here. If you really wanted to dig in you can check the Ruby changelog
Running 2.0 on Heroku
If you’re interested in taking advantage of these new features give it a try on Heroku today. To run Ruby 2.0 on Heroku you'll need this line in your
Then commit to git:
$ git add . $ git commit -m "Using Ruby 2.0 in production"
We recommend that you test your app using 2.0 locally and deploy to a staging app before pushing to production. Now when you
$ git push heroku master our Ruby buildpack will see that you've declared your Ruby version and make sure you get the right one.
20 years of simplicity, elegance, and programmer happiness
Heroku, since its founding, has been aligned with the key values of Ruby – simplicity, elegance, and programmer happiness. Heroku still believes in the power and flexibility of Ruby, and we've invested in the language by hiring Yukihiro "Matz" Matsumoto, Koichi Sasada and Nobuyoshi Nakada. We would like to thank them and the whole Ruby core team for making this release happen. Join us in celebrating Ruby's successes and in looking forward to the next twenty years by trying Ruby 2.0 on Heroku today.
Functional Programming in Java now in beta and PragPub for March now available
In this episode of The Ruby Show, Jason and Peter talk about the latest Ruby release, Rails security releases, and the usual round up of interesting projects.