# Validate JSON schema in Rails ## Topics 1. What/Why JSON schema 2. Apply to rails model validation 3. Test your API endpoint with schema matcher 4. Homework for a curious reader 5. References ### What/Why JSON schema What is JSON schema? I recommend you to read [this](https://brandur.org/elegant-apis) and [that](https://spacetelescope.github.io/understanding-json-schema/index.html) to understand more about JSON schema. However if you are lazy like me, just think JSON schema as the spec of your JSON data, it help you defines how your JSON data should looks like. The simplest schema is like below: ```json { "properties": { "name": { "type": "string" } }, "required": ["name"], "type": "object" } ``` This schema means your JSON object should have a name attribute with string type. for example, this will be a valid JSON: ```json { "name": "Wayne" } ``` However this is not a valid JSON because name is a number but not a string ```json { "name": 5566 } ``` So, why we need JSON schema? What's the benefit? First of all, define your data properly is never a bad idea. And there are at least 4 benefits I can think of: * validate your JSON data structure, so you don't mess up your database * help you validate your APIs response, especially REST-like API * One rule, everywhere. Help your client validate their data. * Bonus: can integrate with Swagger (if you like) Cool, so now let's write some code with our beloved ruby. ### Apply to rails model validation There are 2 gems I found to help me validate JSON Schema, the first one is the ruby implementation of JSON Schema validate called [json-schema](https://github.com/ruby-json-schema/json-schema), the second one is rails validator implementation called [activerecord_json_validator](https://github.com/mirego/activerecord_json_validator) based on first one. Let's use [activerecord_json_validator](https://github.com/mirego/activerecord_json_validator) to integrate our model level JSON validation. Assume we have `User` and `Report`, we allow user to send error report to us including their system's environment and save at `data` column as JSON, migration file looks like below: ```rb # db/migrations/xxxxxxxxx_create_reports.rb class CreateReports < ActiveRecord::Migration def change create_table :reports do |t| t.references :user t.jsonb :data, null: false, default: "{}" t.timestamps null: false end end end ``` We want our Report#data to have at least 2 keys `devise_id` and `version`, so a valid JSON should be like below: ```json { "devise_id": "devise-id-is-a-string", "version": "5.56.6" } ``` To test our model validation, we write rspec code like below: ```rb # spec/models/report_spec.rb RSpec.describe Report, type: :model do describe 'validates data column' do # We use Factory girl to create fake record subject(:report) { create(:report, data: data) } let(:valid_data) do { devise_id: 'devise-id-is-a-string', version: '5.56.6' } end describe 'valid data' do let(:data) { valid_data } it 'creates report' do expect { report }.to change { Report.count }.by(1) end end describe 'invalid data' do context 'when missing devise_id' do let(:data) { valid_data.except(:devise_id) } it 'raise validation error' do expect { report }.to raise_error(ActiveRecord::RecordInvalid) end end context 'when missing version' do let(:data) { valid_data.except(:version) } it 'raise validation error' do expect { report }.to raise_error(ActiveRecord::RecordInvalid) end end end end end ``` Install gem ```sh # Gemfile gem 'activerecord_json_validator' ``` Add validation into `Report` ```rb # app/models/report.rb class Report < ActiveRecord::Base JSON_SCHEMA = "#{Rails.root}/app/models/schemas/report/data.json" belongs_to :user validates :data, presence: true, json: { schema: JSON_SCHEMA } end ``` Add JSON schema file I prefer add `.json` file into `app/models/schemas/report/data.json` ```json // app/models/schemas/report/data.json { "$schema": "http://yourdomain.com/somewhere/report/data", "type": "object", "properties": { "devise_id": { "type": "string" }, "version": { "type": "string" } }, "required": [ "devise_id", "version" ] } ``` Now all test passed. ### Test your API endpoints with schema matcher Now it's time to add some api endpoint response test. Assume we have users api `GET /users` and `GET /users/:id`, lets define our response: ```json // GET /users { "users": [ { "id": 1, "name": "John John Slater", "email": "jjs@example.com", "is_good_surfer": true, "updated_at": "timestamp", "created_at": "timestamp" } ] } // GET /users/:id { "user": { "id": 1, "name": "John John Slater", "email": "jjs@example.com", "is_good_surfer": true, "updated_at": "2017-02-01T10:00:54.326+10:00", "created_at": "2017-02-01T10:00:54.326+10:00" } } ``` Lets write some tests first I'm using Rspec and found out there is a gem called [json_matcher](https://github.com/thoughtbot/json_matchers), it's also based on gem [json-schema](https://github.com/ruby-json-schema/json-schema), so you can choose either one to implement your test, check [this post](https://robots.thoughtbot.com/validating-json-schemas-with-an-rspec-matcher) and you will have more idea how to do this. Ok, time to make our hand dirty. First do some setup: ```rb # Gemfile gem 'json_matchers' # spec/spec_helper.rb require "json_matchers/rspec" # spec/support/json_matchers.rb JsonMatchers.schema_root = "controller/schemas" ``` And write tests ```rb RSpec.describe User, type: :request do describe 'GET /users' do let!(:user) { create(:user) } subject! { get '/users' } specify do expect(response).to be_success expect(response).to match_response_schema('users') end end describe 'GET /users/:id' do let(:user) { create(:user) } subject! { get "/users/#{user.id}" } specify do expect(response).to be_success expect(response).to match_response_schema('user') end end end ``` Test should be failed because we haven't added JSON schema file yet. Add schema file for user, notice we put user object definitions into `definitions` so we can reuse when we define `users.json` ```json // app/controllers/schemas/user.json { "type": "object", "required": ["user"], "properties": { "user": { "$ref": "#/definitions/user" } }, "definitions": { "user": { "type": "object", "required": [ "id", "name", "email", "is_good_surfer", "updated_at", "created_at" ], "properties": { "id": { "type": "integer" }, "name": { "type": "string" }, "email": { "type": "string" }, "is_good_surfer": { "type": "boolean" }, "updated_at": { "type": "string" }, "created_at": { "type": "string" } } } } } ``` ```json // app/controllers/chemas/users.json { "type": "object", "required": ["users"], "properties": { "users": { "items": { "$ref": "user.json#/definitions/user" }, "type": "array" } } } ``` In here we've used a technic called `reference`, as you can see we put `user` object's schema into `definitions` inside `app/controllers/schemas/user.json` and we use `"$ref": "#/definitions/user"` to reference it, it's a best practice to move your schema under definition key so that you can reuse it in the future (even cross file), as you see we also use `"$ref": "user.json#/definitions/user"` inside our `app/controllers/chemas/users.json`, just think it as rails view partial and you'll get it. :sunglasses: Let's run our test, now test passed. As so once again, the day is saved thanks to JSON schema. :joy: Hope you had a good reading. ### Homework for a curious reader If this post can't fulfill your curiosity, there are more topics of JSON schema you can chasing for: * How to write generic JSON schema test and apply to every api endpoints? * How to expose your JSON schema so you can share/use it either at rails project or other repos ### References Also, you can check those reference for more details. Cheers! * https://brandur.org/elegant-apis * https://robots.thoughtbot.com/validating-json-schemas-with-an-rspec-matcher * https://github.com/mirego/activerecord_json_validator * https://github.com/ruby-json-schema/json-schema * https://robots.thoughtbot.com/validating-the-formkeep-api * https://spacetelescope.github.io/understanding-json-schema/ * http://jsonschema.net/#/