Validate JSON schema in Rails

Topics

What/Why JSON schema
Apply to rails model validation
Test your API endpoint with schema matcher
Homework for a curious reader
References

What/Why JSON schema

What is JSON schema?

I recommend you to read this and that to understand more about JSON schema.

However if you are lazy like me, just think JSON schema as the spec of your JSON data, it help you defines how your JSON data should looks like. The simplest schema is like below:

{
  "properties": {
    "name": {
      "type": "string"
    }
  },
  "required": ["name"],
  "type": "object"
}

This schema means your JSON object should have a name attribute with string type.

for example, this will be a valid JSON:

{
  "name": "Wayne"
}

However this is not a valid JSON because name is a number but not a string

{
  "name": 5566
}

So, why we need JSON schema? What's the benefit?

First of all, define your data properly is never a bad idea. And there are at least 4 benefits I can think of:

validate your JSON data structure, so you don't mess up your database
help you validate your APIs response, especially REST-like API
One rule, everywhere. Help your client validate their data.
Bonus: can integrate with Swagger (if you like)

Cool, so now let's write some code with our beloved ruby.

Apply to rails model validation

There are 2 gems I found to help me validate JSON Schema, the first one is the ruby implementation of JSON Schema validate called json-schema, the second one is rails validator implementation called activerecord_json_validator based on first one.

Let's use activerecord_json_validator to integrate our model level JSON validation.

Assume we have User and Report, we allow user to send error report to us including their system's environment and save at data column as JSON, migration file looks like below:

# db/migrations/xxxxxxxxx_create_reports.rb
class CreateReports < ActiveRecord::Migration
  def change
    create_table :reports do |t|
      t.references :user
      t.jsonb :data, null: false, default: "{}"
      t.timestamps null: false
    end
  end
end

We want our Report#data to have at least 2 keys devise_id and version, so a valid JSON should be like below:

{
  "devise_id": "devise-id-is-a-string",
  "version": "5.56.6"
}

To test our model validation, we write rspec code like below:

# spec/models/report_spec.rb
RSpec.describe Report, type: :model do
  describe 'validates data column' do
    # We use Factory girl to create fake record
    subject(:report) { create(:report, data: data) }
    let(:valid_data) do
      {
        devise_id: 'devise-id-is-a-string',
        version: '5.56.6'
      }
    end

    describe 'valid data' do
      let(:data) { valid_data }
      it 'creates report' do
        expect { report }.to change { Report.count }.by(1)
      end
    end

    describe 'invalid data' do
      context 'when missing devise_id' do
        let(:data) { valid_data.except(:devise_id) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end

      context 'when missing version' do
        let(:data) { valid_data.except(:version) }

        it 'raise validation error' do
          expect { report }.to raise_error(ActiveRecord::RecordInvalid)
        end
      end
    end
  end
end

Install gem

# Gemfile
gem 'activerecord_json_validator'

Add validation into Report

# app/models/report.rb
class Report < ActiveRecord::Base
  JSON_SCHEMA = "#{Rails.root}/app/models/schemas/report/data.json"

  belongs_to :user

  validates :data, presence: true, json: { schema: JSON_SCHEMA }
end

Add JSON schema file

I prefer add .json file into app/models/schemas/report/data.json

// app/models/schemas/report/data.json
{
  "$schema": "http://yourdomain.com/somewhere/report/data",
  "type": "object",
  "properties": {
    "devise_id": {
      "type": "string"
    },
    "version": {
      "type": "string"
    }
  },
  "required": [
    "devise_id",
    "version"
  ]
}

Now all test passed.

Test your API endpoints with schema matcher

Now it's time to add some api endpoint response test.

Assume we have users api GET /users and GET /users/:id, lets define our response:

// GET /users
{
  "users": [
    {
      "id": 1,
      "name": "John John Slater",
      "email": "[email protected]",
      "is_good_surfer": true,
      "updated_at": "timestamp",
      "created_at": "timestamp"
    }
  ]
}

// GET /users/:id
{
  "user": {
    "id": 1,
    "name": "John John Slater",
    "email": "[email protected]",
    "is_good_surfer": true,
    "updated_at": "2017-02-01T10:00:54.326+10:00",
    "created_at": "2017-02-01T10:00:54.326+10:00"
  }
}

Lets write some tests first

I'm using Rspec and found out there is a gem called json_matcher, it's also based on gem json-schema, so you can choose either one to implement your test, check this post and you will have more idea how to do this.

Ok, time to make our hand dirty.

First do some setup:

# Gemfile
gem 'json_matchers'

# spec/spec_helper.rb
require "json_matchers/rspec"

# spec/support/json_matchers.rb
JsonMatchers.schema_root = "controller/schemas"

And write tests

RSpec.describe User, type: :request do
  describe 'GET /users' do
    let!(:user) { create(:user) }
    subject! { get '/users' }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('users')
    end
  end

  describe 'GET /users/:id' do
    let(:user) { create(:user) }
    subject! { get "/users/#{user.id}" }

    specify do
      expect(response).to be_success
      expect(response).to match_response_schema('user')
    end
  end
end

Test should be failed because we haven't added JSON schema file yet.

Add schema file for user, notice we put user object definitions into definitions so we can reuse when we define users.json

// app/controllers/schemas/user.json
{
  "type": "object",
  "required": ["user"],
  "properties": {
    "user": {
      "$ref": "#/definitions/user"
    }
  },
  "definitions": {
    "user": {
      "type": "object",
      "required": [
        "id",
        "name",
        "email",
        "is_good_surfer",
        "updated_at",
        "created_at"
      ],
      "properties": {
        "id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        },
        "email": {
          "type": "string"
        },
        "is_good_surfer": {
          "type": "boolean"
        },
        "updated_at": {
          "type": "string"
        },
        "created_at": {
          "type": "string"
        }
      }
    }
  }
}

// app/controllers/chemas/users.json
{
  "type": "object",
  "required": ["users"],
  "properties": {
    "users": {
      "items": {
        "$ref": "user.json#/definitions/user"
      },
      "type": "array"
    }
  }
}

In here we've used a technic called reference, as you can see we put user object's schema into definitions inside app/controllers/schemas/user.json and we use "$ref": "#/definitions/user" to reference it, it's a best practice to move your schema under definition key so that you can reuse it in the future (even cross file), as you see we also use "$ref": "user.json#/definitions/user" inside our app/controllers/chemas/users.json, just think it as rails view partial and you'll get it. 😎

Let's run our test, now test passed.

As so once again, the day is saved thanks to JSON schema. 😂

Hope you had a good reading.

Homework for a curious reader

If this post can't fulfill your curiosity, there are more topics of JSON schema you can chasing for:

How to write generic JSON schema test and apply to every api endpoints?
How to expose your JSON schema so you can share/use it either at rails project or other repos

References

Also, you can check those reference for more details. Cheers!

wayne5540/rails-json-schema-validation.md