QuickBooks — A Platform Transformation — Part 3

Schema driven API development

In Part 2 , we looked the key capabilities required to build a Platform. In Part 3 we will look at how we are building a unified set of APIs that are available for 1st (Internal),2nd (Strategic Partners) and 3rd Party (External) developers.

To solve seamlessly for internal developers who generally are building experiences for mobile first products — and need to optimize chattiness — and for external developers who are familiar with REST (.NET, Java, Python, etc.) we are taking a schema driven approach, supporting multiple interaction models — once we author the specification, the provider, consumer and administrative aspects are programmatically generated.

API Schema
Schemas define the structure of data in the resource model. Services expose a public interface to consumers as described by a resource model. A resource model includes a URL path, e.g. /Customer, a set of interactions (read, write, react and master), and a set of properties, e.g. last name. The read and write interactions translate to the HTTP GET and POST methods. The react interaction indicates that the service is interested in changes to the specified resource. The master interaction indicates that the service has authority over modifications to the specified resource. Internally, the services persist data using a data model. A data model includes entities, their attributes, and their relationships with other entities.

We are leveraging Open API Specification — YAML with JSON specification — to author a resource model. It includes a strong type system and a good way to model entities, properties and constraints. It captures metadata for entities, relationships to other entities, service routing, properties. Properties in turn has a type, name, version and scope (public, private, etc.) and constraints. Finally, rich syntactic constraints are modeled using JSON Schema Validation Specification. .

Providers — On the provider side, we generate basic service interfaces in stubs with default implementation — this uses the container and libraries of choice from Springboot — or any of the JAX RS 2.0 specification frameworks like Apache CXF, Jersey, Resteasy, etc. The service developer then simply implements the domain specific logic. The Mock server porotypes the API and starts exposing it to clients for both feedback and consumption. The service tests are generated through Karate. Finally, we are investing in building a scaffolding for services (a hello QBO service) that will enable developers to be up and running in <15 mins, and continuously deploy code in AWS using Kubernetes. Developers focus on their “app logic” while the “hello QBO” service comes pre-built with all common code and dependencies needed to be deployed in AWS with complete CI/CD.

Consumers — Consumption happens through code generation of SDK, API Docs and Samples. We have built a meta SDK code generator that does lexical analysis of the schema to build an abstract syntax tree (AST) to dynamically generate SDKs for each language — Java, .NET and PHP. The SDK abstracts the details of creating a context, calling the API Gateway and exposing the response back to the application for consumption.

Variability

We have been leveraging Variability engineering as a standard way on how we are architecting our products and that is helping us penetrate new markets quickly. Variability is the ability of a APIs to be efficiently extended, changed, customized, or configured for use in a context. This usually consists of identification and analysis of common and varying parts in a domain model. Customization is a mechanism where a variant feature is implemented via code and configuration is a variation mechanism where a variant feature is implemented through data changes. We are spending time in modeling API variability so that we can reuse strategies based on context and behavior.

Variability of the schema is injected at runtime based of

• Country or geographic region — e.g. what information is required to onboard a new Employee for Payroll varies in each of the 50 US states. By modeling this as configuration we can dynamically drive the end to end experience.
• Type of the customer — we can provide differentiated treatment to our high value customers.

API Design

To unify the needs for both internal and external developers there are some key considerations which we are using to architect our APIs

• Ability to deal with linear entities and nested graphs.
• Ability to deal with interactive experiences.
• Reduce chattiness in mobile experiences through batch interaction requests.
• Ability to support standard REST protocols familiar to 3rd party developers and GraphQL for internal developers.

Generalization of Batch, GraphQL and REST results in a Batch interaction model between the API Gateway and the Domain Services. The Gateway has intelligence to route the graph request to different entities. In the example above Customer and Reporting may get routed to two different domain services.

To conclude , our schema driven approach helps us programmatically generate API stubs, mock tests , service tests and SDKs . Our novel approach unifies the needs for both cross device consumption in building modern experiences while expose a coherent set of APIs to external developers.

In Part 4 we will look at what are key operational measures for success.