!Oh, CRUD!

or, Being Resourceful in REStful API Design

To kick off the API Design Patterns series, we’ll tackle what is likely the most common (and perhaps most maligned) API design pattern, implementing CRUD (or CRUDL) resource operations in an API.

First, I’ll give my definition of the design pattern, then I’ll discuss some of the criticism of the pattern and cogitate on why it prevails in spite of that criticism.

CRUD is an acronym for “Create, Read, Update and Delete” - four central operations for managing persistent application data in a consistent way. CRUD originated in n-tier software solutions as a pattern for data persistence.

CRUDL extends CRUD with L: the List operation, which results in a more complete pattern of resource access. I suggest also adding an E (for robust Error handling) … but add it to CRUDL, not to CRUD, of course — that would be, well, crude.

I’ve discussed CRUDL[E] in API Design Matters - notably in

• C: Create: Getting Creative with OpenAPI

• R: Read: What am I Getting Out of This?

• U: Update: Keep Me Updated, OK?

• D: Delete. I wrote this up, but accidentally deleted it before posting it….

• L: List: From Domain Model to OpenAPI

• E: Errors: Your API Has Problems. Deal With It.

As I said, CRUD and CRUDL “get no respect” in API circles. Some will say a RESTful API should not use CRUDL. I agree, but primarily in the case where CRUDL is the wrong API design pattern to use.

As I’ve mentioned before, the goal of an API is to enable application developers to solve real problems that their application users have. I believe many applications are structured around resource management which maps to CRUDL in a natural manner. Many applications let users view a list of resources, create new items within that resource collection, view (read) the details of an item in the list, edit those items, and delete items from the list. Thus, the conceptual model that many application developers cast over their application objects—and the expression of the desired user experience—fits the CRUDL pattern. Thus, an API which provides CRUDL-oriented operations suits their application needs—and that is, after all, the primary goal of good API design.

The goal of good API design is not to be dogmatic and force application developers to adopt a API model that does not fit their mental model of the system.

The largest risk of CRUD is twofold:

1. Directly exposing database tables (or other implementation) details in the API

2. Treating every problem as something that can or should be solved with CRUDL

Let’s dive deeper into each of these.

Directly exposing database tables (or other implementation) details in the API

It is very tempting to use modern tools that let you put a RESTful API on top of a database. Some tools make this very easy — in my opinion, too easy.

I see several problem with this plan.

An API should be an abstraction that hides the implementation tier and its details from the API consumer. By directly exposing your database schema, you tie all client applications to that schema instead of insulating them. Coupling the client to the database schema is very fragile and leaves you with undesirable outcomes:

1. This limits the set of changes you can make in the database without breaking the clients. Exposing the database through an API derived directly from the DB schema violates one of the most important tenets of API design: separation of client and server, so that each can change independently of the other. An API abstraction can (and should) hide back-end changes (such as normalizing or denormalizing or otherwise reorganizing the database—or even replacing the database with a different DB).

2. Database operations (raw CRUD) are a low-level mechanism for managing persistent data. This is not an application-level set of behaviors, so APIs derived from the DB schema make application development harder as they provide the wrong level of abstraction and require applications to assume on a lot of persistence logic. Some frameworks may take on some of this load, but the tight coupling remains.

3. If your goal is really database I/O, then there are better access patterns such as SQL, JDBC, ODBC or object-relational mapping tools. Don’t “reinvent” industry APIs like SQL with thin “RESTful” APIs that likely do a poorer job of database access than the industry standards while also preventing use of tools (like SQL optimization, connection pooling, etc.) that are possible with standard DB interfaces but likely impossible with a RESTful API facade.

Of course, one can counter that an API locks its consumers into a different class of “schema” and access pattern (the API’s model), but at least with an API abstraction layer, a robust implementation can change implementation decisions while preserving the abstraction.

Treating every problem as something that can or should be solved with CRUDL

This is a more subtle risk of over-reliance on the CRUDL pattern, and a perfectly natural by-product of trying to adhere to the core tenets of REST: Representational State Transfer. This risk consists of defining application behavior as state changes.

Consider an API that provides debit or credit card management for a bank or credit union. A common account holder need is to lock or disable a card that they have (temporarily) misplaced. The account holder is not ready to report the card as lost or stolen and request a replacement—they simply cannot locate it at the moment, but hope or expect to find it. However, to protect their account in case they really have lost it, they want to lock the card so someone who finds it cannot use it.

Adopting a strict “representational state transfer” approach, one may be tempted to support this feature by providing a state-carrying property of a card resource: "locked": true means the card is locked and "locked": false means it is not locked. By extension, therefore, one is tempted to implement card locking and unlocking by using a PUT or PATCH operation to change the locked property to true or false, respectively.

I believe such designs to be fragile and incomplete, for several reasons.

1. First, most API resources have many properties which can potentially be updated with PUT or PATCH… Allowing an API consumer to update multiple properties at the same time adds a high level of complexity and state management burden on the consumer… the client has to understand which fields can be changed together and which may not. While patching a property may seem like a simple solution, many complexities of state management can arise as the API evolves over time. Seemingly simple solutions become very hard to evolve and manage.

2. Such actions may require processing well beyond simply updating the value of a property in a database. In this case, the fact that a card is locked must be communicated to the financial institution or card processor, so that card transactions are blocked at ATMs, retail establishments, etc. That is, locking/unlocking a card is much more than just setting a boolean value. It is misleading to bury those behaviors behind PUT or PATCH operations.

3. This use of the CRUDL pattern obfuscates, hides, or simply ignores the behavior of the shared domain model. Recalling our earlier discussion of domain-driven design and the Align/Define/Design/Refine process, our problem and domain analysis tells us “The customer who has misplaced a card wishes to unlock it so it can’t be used by anyone until they unlock it”. The end user’s goal is to lock or unlock the card—their goal is not to change the representation of a card’s "locked" property to true or false. That is, a cleaner API provides the desired business function with dedicated lockCard and unlockCard operations, with well defined and limited scope behavior.

4. API operations, like APIs, should “do one thing and do one thing well”, so overloading an update operation with multiple behaviors limits the composability of the API, and makes it harder to understand and predict the API’s behavior.

5. Specialized operations, such as dedicated lockCard and unlockCard operations, can also be controlled with entitlement management and fine-grained OAuth scopes — security practices that cannot be communicated for the generic PUT or PATCH operations which update several properties.

6. Consider the corresponding API documentation and developer experience of each approach. As a developer building a front-end application, which API would you prefer:

• one that instructs you: “To lock a card, submit a PATCH operation with a request body containing "locked": true”, with additional rules about what else can be changed or not changed in that request, or

• an API which has dedicated lockCard and unlockCard operations

I know I have a preference. Do you?

Summary

CRUDL is a useful API design pattern, but one which can be overused—even abused—easily. I recommend using CRUDL when the domain calls for it, and not overloading it with behaviors masquerading as state changes.