Add partial fetch tracking for Parse objects

Implements a system to track partially fetched Parse objects, including methods to check fetched fields, autofetch missing fields, and support for nested partial fetches via includes. Updates object construction, associations, and query decoding to propagate partial fetch state. Adds comprehensive integration tests for partial fetch behavior. Bumps version to 2.1.0.

Author: AdrianCurtin

CHANGELOG.md

@@ -1,5 +1,21 @@
 ## Parse-Stack Changelog
 
+### 2.1.0
+
+#### Partial Fetch Tracking System
+- **NEW**: Partial fetch tracking for objects fetched with specific `keys` parameter
+- **NEW**: `partially_fetched?` method to check if object was fetched with limited fields
+- **NEW**: `fetched_keys` / `fetched_keys=` methods to get/set the array of fetched field names
+- **NEW**: `field_was_fetched?(key)` method to check if a specific field was included in the fetch
+- **NEW**: Autofetch triggers automatically when accessing unfetched fields on partially fetched objects
+- **NEW**: Nested partial fetch tracking for included objects via `include:` parameter
+- **NEW**: `nested_fetched_keys` / `nested_keys_for(field)` methods for tracking nested object fields
+- **NEW**: `parse_includes_to_nested_keys` helper parses include patterns like `["team.time_zone", "team.name"]`
+- **FIXED**: Objects fetched with `keys:` parameter no longer have dirty tracking for fields with default values
+- **FIXED**: `clear_changes!` now called after `apply_defaults!` to prevent false dirty tracking
+- **IMPROVED**: Before-save hooks can now reliably access unfetched fields (triggers autofetch)
+- **IMPROVED**: Saving partially fetched objects only updates actually changed fields, not default values
+
 ### 2.0.9
 
 - **FIXED**: `Query#where` method now routes through `conditions` to properly handle special keywords like `keys:`, `include:`, `limit:`, etc. when chaining (e.g., `Model.query.where(keys: [...])`)

lib/parse/model/associations/belongs_to.rb

@@ -165,7 +165,9 @@ def belongs_to(key, opts = {})
             # hash, lets try to buid a Pointer of that type.
 
             if val.is_a?(Hash) && (val["__type"] == "Pointer" || val["__type"] == "Object")
-              val = Parse::Object.build val, (val[Parse::Model::KEY_CLASS_NAME] || klassName)
+              # Get nested fetched keys for this field if available
+              nested_keys = nested_keys_for(key)
+              val = Parse::Object.build val, (val[Parse::Model::KEY_CLASS_NAME] || klassName), fetched_keys: nested_keys
               instance_variable_set ivar, val
             end
             val
@@ -189,7 +191,9 @@ def belongs_to(key, opts = {})
             if val == Parse::Properties::DELETE_OP
               val = nil
             elsif val.is_a?(Hash) && (val["__type"] == "Pointer" || val["__type"] == "Object")
-              val = Parse::Object.build val, (val[Parse::Model::KEY_CLASS_NAME] || klassName)
+              # Get nested fetched keys for this field if available
+              nested_keys = nested_keys_for(key)
+              val = Parse::Object.build val, (val[Parse::Model::KEY_CLASS_NAME] || klassName), fetched_keys: nested_keys
             end
 
             if track == true

lib/parse/model/core/fetching.rb

@@ -42,6 +42,11 @@ def fetch!(opts = {})
 
         # take the result hash and apply it to the attributes.
         apply_attributes!(result, dirty_track: false)
+
+        # Clear partial fetch tracking - object is now fully fetched
+        @_fetched_keys = nil
+        @_nested_fetched_keys = nil
+
         begin
           clear_changes!
         rescue => e
@@ -78,14 +83,16 @@ def fetch_object
 
       # Autofetches the object based on a key that is not part {Parse::Properties::BASE_KEYS}.
       # If the key is not a Parse standard key, and the current object is in a
-      # Pointer state, then fetch the data related to this record from the Parse
-      # data store.
+      # Pointer state or was partially fetched, then fetch the data related to
+      # this record from the Parse data store.
       # @param key [String] the name of the attribute being accessed.
       # @return [Boolean]
       def autofetch!(key)
         key = key.to_sym
         @fetch_lock ||= false
-        if @fetch_lock != true && pointer? && key != :acl && Parse::Properties::BASE_KEYS.include?(key) == false && respond_to?(:fetch)
+        # Autofetch if object is a pointer OR was partially fetched
+        needs_fetch = pointer? || partially_fetched?
+        if @fetch_lock != true && needs_fetch && key != :acl && Parse::Properties::BASE_KEYS.include?(key) == false && respond_to?(:fetch)
           #puts "AutoFetching Triggerd by: #{self.class}.#{key} (#{id})"
           @fetch_lock = true
           send :fetch

lib/parse/model/core/properties.rb

@@ -273,8 +273,10 @@ def property(key, data_type = :string, **opts)
 
           # If the value is nil and this current Parse::Object instance is a pointer?
           # then someone is calling the getter for this, which means they probably want
-          # its value - so let's go turn this pointer into a full object record
-          if value.nil? && pointer?
+          # its value - so let's go turn this pointer into a full object record.
+          # Also autofetch if object was partially fetched and this field wasn't included.
+          should_autofetch = value.nil? && (pointer? || (partially_fetched? && !field_was_fetched?(key)))
+          if should_autofetch
             # call autofetch to fetch the entire record
             # and then get the ivar again cause it might have been updated.
             autofetch!(key)

lib/parse/model/object.rb

@@ -311,11 +311,13 @@ def initialize(opts = {})
       # ACL.typecast will auto convert of Parse::ACL
       self.acl = self.class.default_acls.as_json if self.acl.nil?
 
-      clear_changes! if @id.present? #then it was an import
-
       # do not apply defaults on a pointer because it will stop it from being
       # a pointer and will cause its field to be autofetched (for sync)
       apply_defaults! unless pointer?
+
+      # clear changes AFTER applying defaults, so fields set by defaults
+      # are not marked dirty when fetching with specific keys
+      clear_changes! if @id.present? #then it was an import
       # do not call super since it is Pointer subclass
     end
 
@@ -382,6 +384,77 @@ def existed?
       created_at != updated_at
     end
 
+    # Returns whether this object was fetched with specific keys (partial fetch).
+    # When partially fetched, accessing unfetched fields will trigger an autofetch.
+    # @return [Boolean] true if the object was fetched with specific keys.
+    def partially_fetched?
+      @_fetched_keys.present? && @_fetched_keys.any?
+    end
+
+    # Returns the array of keys that were fetched for this object.
+    # Empty array means the object was fully fetched.
+    # @return [Array<Symbol>] the keys that were fetched.
+    def fetched_keys
+      @_fetched_keys || []
+    end
+
+    # Sets the fetched keys for this object. Used internally when building
+    # objects from partial fetch queries.
+    # @param keys [Array] the keys that were fetched
+    # @return [Array] the stored keys
+    def fetched_keys=(keys)
+      if keys.nil? || keys.empty?
+        @_fetched_keys = nil
+      else
+        # Always include :id and convert to symbols
+        @_fetched_keys = keys.map { |k| Parse::Query.format_field(k).to_sym }
+        @_fetched_keys << :id unless @_fetched_keys.include?(:id)
+        @_fetched_keys << :objectId unless @_fetched_keys.include?(:objectId)
+        @_fetched_keys.uniq!
+      end
+      @_fetched_keys
+    end
+
+    # Returns whether a specific field was fetched for this object.
+    # Base keys (id, created_at, updated_at) are always considered fetched.
+    # @param key [Symbol, String] the field name to check
+    # @return [Boolean] true if the field was fetched or if object is fully fetched.
+    def field_was_fetched?(key)
+      # If not partially fetched, all fields are considered fetched
+      return true unless partially_fetched?
+
+      key = key.to_sym
+      # Base keys are always considered fetched
+      return true if Parse::Properties::BASE_KEYS.include?(key)
+      return true if key == :acl || key == :ACL
+
+      # Check both local key and remote field name
+      remote_key = self.field_map[key]
+      fetched_keys.include?(key) || (remote_key && fetched_keys.include?(remote_key))
+    end
+
+    # Returns the nested fetched keys map for building nested objects.
+    # @return [Hash] map of field names to their fetched keys
+    def nested_fetched_keys
+      @_nested_fetched_keys || {}
+    end
+
+    # Sets the nested fetched keys map for building nested objects.
+    # @param keys_map [Hash] map of field names to their fetched keys
+    # @return [Hash] the stored map
+    def nested_fetched_keys=(keys_map)
+      @_nested_fetched_keys = keys_map.is_a?(Hash) ? keys_map : nil
+    end
+
+    # Gets the fetched keys for a specific nested field.
+    # @param field_name [Symbol, String] the field name
+    # @return [Array, nil] the fetched keys for the nested object, or nil if not specified
+    def nested_keys_for(field_name)
+      return nil unless @_nested_fetched_keys.present?
+      field_name = field_name.to_sym
+      @_nested_fetched_keys[field_name]
+    end
+
     # Run after_create callbacks for this object.
     # This method is called by webhook handlers when an object is created.
     # @return [Boolean] true if callbacks executed successfully
@@ -485,8 +558,10 @@ def clear_attribute_change!(atts)
     #   post = Post.build({"title" => "My Title"})
     # @param json [Hash] a JSON hash that contains a Parse object.
     # @param table [String] the Parse class for this hash. If not passed it will be detected.
+    # @param fetched_keys [Array] optional array of keys that were fetched (for partial fetch tracking).
+    # @param nested_fetched_keys [Hash] optional map of field names to their fetched keys for nested objects.
     # @return [Parse::Object] an instance of the Parse subclass
-    def self.build(json, table = nil)
+    def self.build(json, table = nil, fetched_keys: nil, nested_fetched_keys: nil)
       className = table
       className ||= (json[Parse::Model::KEY_CLASS_NAME] || json[:className]) if json.is_a?(Hash)
       if json.is_a?(Hash) && json["error"].present? && json["code"].present?
@@ -501,7 +576,12 @@ def self.build(json, table = nil)
       if klass.present?
         # when creating objects from Parse JSON data, don't use dirty tracking since
         # we are considering these objects as "pristine"
-        o = klass.new(json)
+        o = klass.allocate
+        # Set nested fetched keys BEFORE apply_attributes! so nested objects can use them
+        o.instance_variable_set(:@_nested_fetched_keys, nested_fetched_keys) if nested_fetched_keys.present?
+        o.send(:initialize, json)
+        # Set fetched keys for partial fetch tracking
+        o.fetched_keys = fetched_keys if fetched_keys.present?
       else
         o = Parse::Pointer.new className, (json[Parse::Model::OBJECT_ID] || json[:objectId])
       end

lib/parse/query.rb

@@ -1375,7 +1375,53 @@ def all(expressions = { limit: :max }, &block)
     # @param list [Array<Hash>] a list of Parse JSON hashes
     # @return [Array<Parse::Object>] an array of Parse::Object subclasses.
     def decode(list)
-      list.map { |m| Parse::Object.build(m, @table) }.compact
+      # Pass fetched keys for partial fetch tracking (only if keys were specified)
+      fetch_keys = @keys.present? && @keys.any? ? @keys : nil
+
+      # Parse includes to build nested fetched keys map
+      nested_keys = parse_includes_to_nested_keys(@includes) if @includes.present?
+
+      list.map { |m| Parse::Object.build(m, @table, fetched_keys: fetch_keys, nested_fetched_keys: nested_keys) }.compact
+    end
+
+    # Parses include patterns to build a map of nested fetched keys.
+    # For example, ["team.time_zone", "team.name", "author"] becomes:
+    # { team: [:time_zone, :name], author: [] }
+    # @param includes [Array<Symbol>] the include patterns
+    # @return [Hash] a map of nested field names to their fetched keys
+    def parse_includes_to_nested_keys(includes)
+      return {} if includes.nil? || includes.empty?
+
+      nested_map = {}
+
+      includes.each do |include_path|
+        parts = include_path.to_s.split('.')
+        next if parts.empty?
+
+        # First part is the field name on the parent object
+        field_name = parts.first.to_sym
+
+        # Initialize the array for this field if not already
+        nested_map[field_name] ||= []
+
+        # If there are more parts, they are fields on the nested object
+        if parts.length > 1
+          nested_field = parts[1].to_sym
+          nested_map[field_name] << nested_field unless nested_map[field_name].include?(nested_field)
+
+          # Handle deeper nesting (e.g., team.manager.name)
+          if parts.length > 2
+            # For now, we'll create a key for the second level too
+            # This would need recursive handling for deeper nesting
+            second_level_field = parts[1].to_sym
+            nested_map[second_level_field] ||= []
+            nested_field = parts[2].to_sym
+            nested_map[second_level_field] << nested_field unless nested_map[second_level_field].include?(nested_field)
+          end
+        end
+      end
+
+      nested_map
     end
 
     # Builds Parse::Pointer objects based on the set of Parse JSON hashes in an array.

lib/parse/stack/version.rb

@@ -6,6 +6,6 @@ module Parse
   # The Parse Server SDK for Ruby
   module Stack
     # The current version.
-    VERSION = "2.0.9"
+    VERSION = "2.1.0"
   end
 end