Archive the original file first.
After exporting records from a platform, save the original file—noting the account, time range, export time zone, and generation date—before creating a working copy. Do not simply delete rows that appear to be duplicates; different tables may represent orders, individual trade executions, ledger adjustments, or on-chain withdrawals. Identical formatting does not imply identical events; creating a field dictionary is more reliable than simply clicking "remove duplicates."
Prioritize the use of stable event identifiers.
If transaction, trade, or ledger identifiers are available, combine them with the platform, account, and event type to create a unique key. Never rely solely on timestamp and amount for deduplication, as multiple trades of the same quantity can occur within the same second. Furthermore, identifier namespaces may vary across different platform APIs; verify whether records can be linked across tables. Flag any records lacking stable identifiers for manual review—do not attempt to "guess" or fabricate a seemingly definitive result.
A single order may correspond to multiple trade executions.
Partial fills result in multiple trade prices, quantities, and fees. Importing both order summaries and individual trade executions can lead to the double-counting of traded volumes. Decide which level will serve as your primary ledger, and retain the other for reference. Since fees may be deducted in a different token, always track the specific asset unit for each fee; do not simply subtract all fees from the total quantity purchased.
Distinguish between internal transfers and on-chain withdrawals.
Transfers between a main account and a sub-account may not generate an on-chain transaction hash. Similarly, a movement of funds between self-custody wallets—involving an outflow from one and an inflow to another—should not be automatically classified as two separate transactions. Instead, you should track the source account, destination account, amount, and any supporting documentation; if a match cannot be found, flag it as an exception. Whether a transaction constitutes a taxable event depends on local regulations and the actual nature of the transaction; the deduplication tool itself cannot provide a definitive tax conclusion.
Validate your rules using a small sample set.
Select a few representative transactions—such as a standard purchase, a partially filled order, a withdrawal, and an internal transfer—and manually trace how each is represented across your various data files. Your rules should clearly explain which rows are retained, which fields are merged, and the rationale behind these decisions. When exporting data that covers overlapping time periods, verify that the same event is consistently matched; if a platform updates its historical data, ensure that old files are not silently overwritten.
Maintain a record of discrepancies and an audit trail.
Your final dataset should include the original filename, original row number, applied cleaning rules, time zone conversions, and the reasons for any exceptions. Additionally, perform a reconciliation of transaction amounts against the opening and closing balances of each account. Any unexplained discrepancies should be listed separately rather than forced into a category like "other income" to balance the accounts. This document is intended for data organization purposes only and does not constitute personalized tax advice. For actual tax filings, please consult with a professional familiar with the relevant local regulations.