Computer stuff, Ruby

Rails API and timestamps shenanigans

tl;dr
Create an initializer with this in it.

# config/initializers/time_precision.rb

ActiveSupport::JSON::Encoding.time_precision = 6

Introduction

I was upgrading an API from Rails 6 to Rails 7 the other day, and I ran into a weird issue when migrating timestamps tests in my request specs. Unraveling these failures led me down an interesting rabbit hole, so I thought it would be worthwhile to document it here for later reference.

In particular, I think there may be a small improvement that could be made to Rails itself to reduce friction when it comes to testing timestamps.

Note
All the snippets shown in this article can be found and reproduced from the repository below.
https://gitlab.com/Richard-Degenne/rails-timestamps

The setup

Alright, let's take a minimal example to show exactly what I'm talking about.

Being the awesome developer that you are, you use test-driven development for everything, including request specs. In case, you are unaware, request specs are a feature of RSpec's Rails extension that behave like integration tests for APIs: build a request, run it through your entire stack with little to no mocking or stubbing, and analyze the side-effects or response to the request.

In this example, I have a model called Post that has a title, textual contents and timestamps for creation and last update. I want to ensure that my API renders the objects correctly.

describe "GET /show" do
  subject(:show) do
    get post_url(post), as: :json
    response
  end

  let!(:post) { create(:post) }

  it { is_expected.to have_http_status :ok }

  it 'renders the post' do
    expect(show.parsed_body).to include(
      title: post.title, contents: post.contents,
      created_at: post.created_at, updated_at: post.updated_at # <= This is the interesting part
    )
  end
end

In particular, I want to check that the timestamps created_at and updated_at are present in the response and that their value at consistent with whatever is in the database.

The catch

Obviously, if this was going to work, I wouldn't be writing an article about it. So here is what happens.

Failures:
  1) /posts GET /show renders the post
     Failure/Error:
       expect(show.parsed_body).to include(
         title: post.title, contents: post.contents,
         created_at: post.created_at, updated_at: post.updated_at
       )

       expected {
        "created_at" => "2025-01-02T18:46:26.126Z",
        "updated_at" => "2025-01-02T18:46:26.126Z",
       }
       to include {
        :created_at => 2025-01-02 18:46:26.126823000 UTC +00:00,
        :updated_at => 2025-01-02 18:46:26.126823000 UTC +00:00
       }
       Diff:
       @@ -1,5 +1,7 @@
       -:created_at => 2025-01-02 18:46:26.126823000 +0000,
       -:updated_at => 2025-01-02 18:46:26.126823000 +0000,
       +"created_at" => "2025-01-02T18:46:26.126Z",
       +"updated_at" => "2025-01-02T18:46:26.126Z"

What exactly is going on? RSpec's include matcher, for each expected key-value pair provided, retrieves the value in the actual hash (in this case, show.parsed_body) and performs an equality test between the expected value and the actual value.

The actual value her is post.created_at, an instance of <a href="https://api.rubyonrails.org/classes/ActiveSupport/TimeWithZone.html">ActiveSupport::TimeWithZone</a>, and its comparison operator relies on <a href="https://ruby-doc.org/core-3.0.0/Time.html#method-i-3C-3D-3E">Time#<=></a>, which is able to perform casts for the second operand. We can check this with some tests.

str = '2025-01-02T12:00:00'
time = Time.zone.parse(str)
time == str

# => true

That means that the JSON value being a String should not prevent the comparison from working. There is something more at play here...

The rabbit hole

Taking a closer look at the failure message, I noticed something strange though: the number of decimal places. Even though the database timestamp are precise to the microsecond, the timestamp rendered in the JSON response is truncated down to the millisecond. Obviously, this is certain to throw a wrench in the comparison, since the truncation loses part of the information.

str = '2025-01-02T12:00:00.123456'
time = Time.zone.parse(str)
time == '2025-01-02T12:00:00.123'

# => false

Now, the question becomes this: what is responsible for rendering the timestamps, and why does it truncate them to the millisecond? Well, as it turns out, Rails relies on the ActiveSupport::JSON module to transform any and all Ruby values into JSON, through the <a href="https://github.com/rails/rails/blob/cf6ff17e9a3c6c1139040b519a341f55f0be16cf/activesupport/lib/active_support/core_ext/object/json.rb">as_json</a> helper that's monkey-patched onto basically all Ruby objects. Let's see what that looks like for timestamps.

time = Time.zone.parse('2025-01-02T18:00:00.123456')
time.as_json

# => "2025-01-02T18:00:00.123Z"

Hah! Here is our unexpected truncation! I feel like this behavior could be considered surprising at best, and there is even a good argument to consider that a bug, because almost every other implementation of as_json conserves the equality.

42 == 42.as_json
# => true

nil == nil.as_json
# => true

'richard' == 'richard'.as_json
# => true

# The only *obvious* exception I can think of are Symbols, that get transformed into strings, breaking the equality.
:richard == :richard.as_json
# => false

Let's take a look at the way <a href="https://api.rubyonrails.org/classes/ActiveSupport/TimeWithZone.html#method-i-as_json">ActiveSupport::TimeWithZone#as_json</a> is implemented as of Rails 8.0.1:

def as_json(options = nil)
  if ActiveSupport::JSON::Encoding.use_standard_json_time_format
    xmlschema(ActiveSupport::JSON::Encoding.time_precision) # <= This is it!
  else
     %(#{time.strftime("%Y/%m/%d %H:%M:%S")} #{formatted_offset(false)})
  end
end

This <a href="https://github.com/rails/rails/blob/cf6ff17e9a3c6c1139040b519a341f55f0be16cf/activesupport/lib/active_support/json/encoding.rb#L107">ActiveSupport::JSON::Encoding.time_precision</a> value is a setting that defaults to 3, meaning that all timestamps are truncated down to the millisecond by default, no matter what the precision of the underlying timestamp is! What?! Why?!

This doesn't make any sense to me. Nowadays, ActiveRecord creates all timestamps columns with a precision of 6 by default, which is why the timestamps in the examples above have microseconds. The decision not to render the full timestamp is just weird.

The fix

At least, we now have an easy way to fix the problem and have all timestamps rendered with the same precision as the underlying database values. We can simply create an initializer where we set this ActiveSupport::JSON::Encoding.time_precision to 6.

# config/initializers/time_precision.rb

ActiveSupport::JSON::Encoding.time_precision = 6

And, sure enough, this fixes our janky comparison, which, in turn, fixes the faulty request spec.

time = Time.zone.parse('2025-01-02T18:00:00.123456')
time.as_json
# => "2025-01-02T18:00:00.123456Z"

time == time.as_json
# => true
/posts
  GET /show
    is expected to respond with status code :ok (200)
    renders the post

Finished in 0.61558 seconds (files took 0.95531 seconds to load)
2 examples, 0 failures

Historical side-note

The last mystery for me was that I already had these tests before and I wasn't having any of these errors. After looking into it for a while, it turns out that back in the day, we're talking Rails 5 here, ActiveRecord created timestamps as datetime columns, without any precision, meaning that the database values were rounded down to the second. With the value's precision being lower than the arbitrary ActiveSupport encoding precision, the comparison with JSON-encoded values held up.

time = Time.current.floor
time.as_json
# => "2025-01-02T21:34:22.000Z"

time == time.as_json
# => true

It also means that ActiveSupport used to render unnecessary zeroes on every single timestamp, but hey, I guess that's not as important.

Improving Rails?

So, the main takeaway from all of this is that I don't really understand why ActiveSupport imposes an arbitrary truncation on timestamps that does not coincide with any other default in Ruby or Rails. I believe that 6 should be the default to match the default ActiveRecord precision, or even 9 to match the default Ruby Time precision.

Another approach could be extending ActiveSupport::TimeWithZone so that each instance could carry its own precision. That way, when ActiveRecord retrieves datetimes from the database, it could embed the column's precision inside the Ruby times, helping providing more meaningful comparison, rounding, serialization...

There is also an argument to be made for a more dynamic approach that detects the last non-zero sub-second digit and stops rendering there. However, looking at <a href="https://ruby-doc.org/core-3.0.0/Time.html#method-i-strftime">Time#strftime</a> internals, and even C's strftime function, it seems that dynamic sub-second formatting has never been a thing.

A question, a feedback? Write a comment!