A successful API response does not guarantee a complete dataset.
A successful API response only indicates that a request was processed; the results may be constrained by pagination, time ranges, data retention periods, or access permissions. Before exporting transaction records, clarify whether the API returns orders, executed trades, or ledger entries, and verify the scope regarding accounts, sub-accounts, and products. Never assume the first page contains the entire history, and do not interpret an empty array as proof that an account has never been active.
Pagination must be implemented strictly according to the documentation.
Different APIs may use time, event identifiers, or cursors (forward/backward) for pagination; sorting logic and boundary inclusion rules can also vary. Store the parameters, response ranges, and the next cursor for every request—do not simply assume that adding one second to a timestamp will prevent data loss. Multiple records can occur within the same millisecond; while stable identifiers are generally more reliable for tracking than timestamps alone, their specific scope must still be verified.
Use overlapping windows to detect issues.
Allow for slight overlaps between adjacent time windows, then deduplicate records using unique event keys. This helps verify boundary conditions; do not use simple deduplication to mask changes in data fields. For instance, if multiple trades occur at the same timestamp, a window-splitting approach that only retains the first record will result in missing data. Record the earliest and latest timestamps, the number of records returned, and any changes to the cursor; if the same page appears repeatedly, stop the process and investigate.
Rate limiting and retries must be handled in a recoverable manner.
Perform requests in batches in accordance with official rate limits. Implement backoff-and-retry logic for rate limits or transient failures, and resume from the last confirmed cursor position. Do not request unnecessary API permissions in an attempt to increase speed—under no circumstances should read-only keys be upgraded to include withdrawal privileges. API keys and signatures must never be included in logs; recording a summary of each request is sufficient for auditing purposes. Log files should also be protected to prevent the exposure of account-identifying information.
Ensure data integrity through cross-verification.
Reconcile exported trade volumes, fees, and asset changes against account statements, monthly reports, or opening and closing balances. Since inflows, outflows, rewards, and adjustments may originate from other API endpoints, trade records alone may not account for the entire balance. Flag any unexplained discrepancies; do not fabricate "balancing" transactions. If historical data exceeds the API’s retention period, consult official statements or submit a support ticket.
Produce an auditable data package.
Document the API version, export timestamp, account scope, pagination parameters, record count, deduplication rules, and a list of exceptions; also, maintain secure copies of the raw API responses. These records should be sufficient to verify any integrity claims made by third-party tools. This document is intended for data organization purposes; it does not guarantee the long-term availability of any specific API, nor does it constitute a tax conclusion or an assessment of a platform’s solvency.