What is going on in there?

From the dawn of science, the question always comes back: “What is going on in there?” How fast is the anesthetic going into the patient? What propulsion systems are being fired by the rocket? What pages are shoppers looking at? And now that software is eating the world, what is going on inside that application?

In any business, it is critical to know how business is going - how many people are coming to your site? How long do they stay? What attracts their attention? What leaves them indifferent? How often do they transact, and for how much? In a startup, it can mean the difference between taking off and dying off.

Urbandoor is an online marketplace that matches companies looking for serviced apartments with providers of such apartments and lets them transact quickly and easily. We have three main categories of Customers. First, there are Relocation Management Companies (RMC's), Travel Management Companies (TMC's) or small businesses booking on behalf of guests. Second, the guest themselves who sometimes book, and always require a smooth end-to-end experience from confirmation to check out. Finally, there are the Providers who work on our platform and are our partners and customers all at once. Our mission is to keep all three happy at all times.

Business Events for Customer Success

One of the most critical moments for us is when a customer or a provider has just been onboarded and as a result, they are new to the platform. We have a Customer Success team whose sole job is to make sure that Customers and Providers have smooth, successful transactions. One of the tools we built to assist this is an alert system that lets us know when recently onboarded Customers and Providers are using the platform, and what they are doing.

As I hope this post is both interesting and informative (going so far as to show others how to implement similar tools), I want to spell out some important privacy considerations before going any further. First, any data that is collected about the customer that is personally identifiable should be kept in a secure database that is only accessible by authorized individuals in your company, and your company only. The specific folks who get access to the business events should be your customer success team (or support team), who need this information to better support your customers and your partners.

Second, your customers and partners should be fully aware of how you work to support them and how you safeguard their privacy in clear FAQs and Terms of Service. You can see a copy of Urbandoor's Privacy Policy here.

Back to Business Events...

Because we use Business Events for support purposes, we need to know that they are happening as they are happening - in real time. In this sense ,they are radically different from traditional Business Intelligence, where a system might run every hour or every day and generate all manners of statistics and KPIs (Key Performance Indicators - the metrics a business or unit tracks). For support, you need to know now.

The obvious solution for communicating business-critical information immediately is to use a business messaging tool, such as Slack, Skype for Business or Cisco Spark. We use Slack for collaboration, so we decided to surface Business Events there.

Whenever a recently-onboarded customer logs in, performs a search or sends a request to a provider on our platform, a Business Event gets surfaced in the system. Additionally, as our marketplace continues to grow, notifications for new properties being added also show up in our Customer Success team’s Slack channel.

When a Provider responds with an offer, another Business Event gets created. If anything goes wrong along the way - perhaps the Customer's search returned fewer properties than it should have, or the provider does not answer in a timely fashion - our Customer Success team can quickly reach out, educate, and help move things along.

In no time at all, our Customers and Providers are able to successfully use the platform without help from our Customer Success team, and we get out of the way. But like anyone who's tried a new service or app, sometimes you just want a little help getting started.

Business Intelligence

We have thousands of properties on our marketplace, and some of the biggest names in corporate America on our customer list. While it makes sense to track specific activity for support purposes, it is also important to monitor trends and data in the aggregate. This is what usually known as Business Intelligence (BI). We use BI to answer questions like “What are the most popular cities from January to March?” or  “Do the average response times vary between large and small providers?”

We are using Business Events to build up an image of our users’ activities, preferences, likes and dislikes. Business Intelligence is different from Business Events in that they are stripped of PII (Personally Identifiable Information).

Business Intelligence is a vast topic by itself and will be the subject of future blog posts.

How It All Works

This is the technical section. Delve in at your own risk!

Event Manager to Segment

We use Ruby on Rails for our core business logic, and at the core of our system is an Event Manager. The Manager creates an event whenever an important business event happens: a user logs in, performs a search, books a stay, arrives at a property, etc. These events are queued up and asynchronously sent to Segment (https://segment.com/), a nice, modern service that takes incoming events and propagates them for us. It propagates them to one of two types of destination, or both:

  • There are many Integrations, which include Slack (the one we used), but also MixPanel, Google Analytics, Send with Us, Mail Chimp, Marketo, Periscope, Salesforce, Zendesk and many more.

  • A data warehouse: either your own PostgreSQL database or your AWS Redshift cluster.

Integrating with Segment is child’s play. We used the analytics-ruby gem (https://github.com/segmentio/analytics-ruby), which gives you full, asynchronous access to Segment’s API. There are three calls that matter:

  • Identify your users uniquely (identify)

  • Add information about your users (traits)

  • Track events (track)

Each event supports multi-level JSON. For example an event might look like this:

event: {

    user_id: 123456

    event_type: user_loggedIn

    properties: {

        agent: Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0)

traits: {

                email: foo@bar.com

                full_name: Foo B. Baz

                company_id: 654321

                }

        }

    }

This means you can track virtually anything that is relevant to your business. In our case, we keep track of the User Agent, so that we know what browsers are most popular with our customers. This allows us to allocate testing resources more intelligently.

Readers with a keen eye will have noticed that we replicated user traits in business event payloads. Since we track traits elsewhere and can get back to it with a simple query, this seems redundant. Unfortunately, some integration, such as the Slack integration, does not support traits yet, so we must add them here redundantly. But disk space is cheap, and de-normalization is the order of the day!

We always use a systematic structure of camelCaseObject_camelCaseAction, delimited by an underscore: user_loggedIn, offer_received, booking_completed, etc. All property fields are lowercase and underscore-separated. These practices have several benefits:

  • They work well with databases; these properties will ultimately turn into column names and values and you want to avoid characters that are special in SQL.

  • The systematic structure of the event types allows us to easily reason about all events where a user did something because we can query for all events where the type is ‘booking_*’ and reason about all booking events.

Conclusion and Next Steps

We’ve found the use of business events to be a valuable addition to our business, giving us the right visibility into our business activity.

In our upcoming blog posts we may tackle:

  • Extending our Segment Integration to use MixPanel.

  • Using Amazon Redshift as a Business Intelligence data warehouse

  • Business events to drive business logic: how an “Event Bus” approach allows for the decoupling of business logic components across packages and processes

Thanks for reading!