How long does a typical data migration project take from start to finish?

Timeline varies dramatically based on data volume, source system complexity, and acceptable risk tolerance. A straightforward migration of 500,000 records from a well-documented SQL database might complete in 8-10 weeks including testing. Complex migrations from legacy systems like AS/400 or FoxPro with millions of records, multiple integrations, and strict compliance requirements typically require 4-6 months. The parallel run period alone—where both systems operate simultaneously—usually lasts 30-60 days depending on your business cycle. We can compress timelines by increasing risk, but we explicitly document what validation steps get shortened and what failure modes become more likely.

What happens if we discover major data quality problems during migration?

Data quality issues are nearly universal—we've never encountered a legacy system without some level of duplicates, constraint violations, or inconsistent formatting. This is exactly why we start with comprehensive profiling rather than jumping into migration. When we discover quality problems, we present options: clean the source data before migration, implement transformation rules that clean during migration, or accept the data as-is with documented risks. For one client, we found 18,000 duplicate customer records; they chose to deduplicate in the source system over six weeks rather than carry the problem forward. Our profiling phase exists specifically to surface these issues when you still have options rather than during a cutover crisis.

Can we migrate incrementally or does everything move at once?

Incremental migration is often the lower-risk approach, and our architecture supports it explicitly. We've migrated organizations one warehouse at a time, one fiscal year at a time, one department at a time—whatever logical division minimizes operational disruption. A distribution company with seven locations migrated one branch every week, allowing us to refine the process and catch issues when they affected 50 users instead of 350. The tradeoff is longer project duration and more complex synchronization logic to keep migrated and non-migrated data consistent. We help you analyze whether incremental or big-bang approach makes sense for your specific situation, considering risk tolerance, budget, and operational constraints.

How do you handle migrations from very old legacy systems like AS/400 or FoxPro?

Legacy platform migrations are actually our specialty—we have developers on staff who've worked with AS/400, Visual FoxPro, Paradox, and even custom flat-file formats from the 1990s. The key challenges are understanding data type conversions (how AS/400 packed decimal fields map to SQL Server), character encoding (especially for data that predates Unicode), and extracting business logic that's embedded in the platform itself. We build custom extraction tools rather than relying on generic ETL products that don't understand these older systems. For one AS/400 migration, we discovered that date fields used a Julian date format with a two-digit year, requiring specific transformation logic to prevent Y2K-style errors. This isn't work we outsource—it's hands-on development by people who've seen these problems before.

What's your approach to maintaining integrations during migration?

Integration preservation starts with comprehensive mapping of every system that touches the source database—we document APIs, database views, export files, scheduled jobs, and even Excel files with ODBC connections. Each integration gets classified by criticality and modification flexibility. High-criticality, inflexible integrations (like connections to vendor systems you can't modify) get API preservation layers that make the new system look identical to the old one from the integration's perspective. Lower-criticality integrations might be updated to use new endpoints. We test every integration in staging environments before production cutover. For one client with 23 integrations, we built seven different preservation layers because the connected systems were owned by different departments with varying update capabilities.

How do you ensure compliance requirements are met during migration?

Compliance varies by industry but generally requires audit trails, data integrity proof, and validation documentation. We implement database-level audit logging that tracks every record change with timestamp and user information. Our validation framework generates reports showing that migrated data matches source data within defined tolerances. For healthcare clients, we ensure test environments with production data are HIPAA-compliant with proper encryption and access controls. Financial services clients get complete chain-of-custody documentation for all data transformations. We've worked with clients' compliance officers to define exactly what evidence they need, then build validation processes that generate that evidence automatically. The goal is making compliance documentation a byproduct of the migration process rather than an additional burden.

What's included in your post-migration support and monitoring?

Our standard engagement includes 90 days of post-cutover monitoring with automated daily reconciliation reports comparing the new system against archived source data. We track record counts, aggregate values (total revenue, inventory quantities, etc.), and business-critical calculations to detect divergence early. Weekly status calls review any discrepancies and user-reported issues. We maintain on-call availability for critical problems and provide priority support for any data integrity questions. This monitoring has caught issues in 23% of our migrations that appeared only after specific business scenarios occurred—like month-end close processes or quarterly reporting cycles. After 90 days, we transition to your internal IT team with complete documentation and source code, though many clients retain us for ongoing [database services](/services/database-services) support.

How much does data migration typically cost?

Migration costs vary based on data volume, source system complexity, number of integrations, and required parallel run duration. A straightforward migration of a well-documented database with 100,000 records and minimal integrations might start around $45,000. Complex migrations from legacy platforms with millions of records, extensive data cleaning requirements, and multiple integrated systems typically range from $125,000 to $350,000. The real cost drivers are data quality issues discovered during profiling, integration complexity, and compliance requirements. We provide fixed-price quotes after the initial discovery phase when we understand actual data conditions. It's worth noting that failed migrations typically cost 2-3x what a properly executed migration would have cost due to business disruption, emergency remediation, and potential data loss recovery.

Can you migrate data to cloud platforms like Azure SQL or AWS RDS?

Yes, cloud platform migrations are increasingly common and introduce specific considerations around network bandwidth, data transfer costs, and security during transit. We've migrated clients from on-premises SQL Server to Azure SQL Database, from legacy AS/400 to AWS RDS, and from various source platforms to cloud data warehouses. The validation framework remains identical, but we add considerations for optimizing data transfer (sometimes shipping encrypted drives is faster than network transfer for multi-terabyte datasets), configuring VPNs for parallel run synchronization, and ensuring compliance with data residency requirements. Cloud migrations often include modernization opportunities—moving to managed services, implementing better backup strategies, or leveraging cloud-native features that weren't available on-premises.

What makes your migration approach different from using standard ETL tools?

Standard ETL tools like SSIS or Informatica are excellent for ongoing integration work but have limitations for one-time migrations, especially from legacy platforms. Our approach uses custom-written transformation code in C# and SQL that explicitly documents every business rule and data mapping decision. This code becomes your permanent documentation of what happened during migration—two years later, you can review the source code to understand exactly why a field was transformed a certain way. ETL tools create black-box transformations that are difficult to audit and can't handle edge cases from legacy systems (like AS/400 packed decimals or FoxPro memo fields). We also build validation logic directly into the transformation process rather than as a separate step, catching errors immediately. That said, we do use ETL tools when they're the right choice—our goal is successful migration, not dogmatic adherence to a particular technology approach.

Solution

Data Migration Services: Clean, Transform & Move Your Data Safely

Enterprise data migration with full cleansing, schema mapping, transformation, and rollback planning — from a Zeeland, Michigan company with 20+ years moving data between legacy databases, modern platforms, and cloud environments. SQL Server, Oracle, PostgreSQL, MySQL, AS/400, Access, and everything in between.

20+ Years Enterprise Migration

150+ Migration Projects

Zero Data Loss Record

Zeeland, MI

Why Data Migrations Fail (And How to Prevent It)

Gartner estimates that 83% of data migration projects either fail outright or exceed their budget and timeline. That number has held steady for over a decade, and the reasons are consistent: poor data quality in the source system, incomplete schema mapping, no rollback plan, and testing that covers the happy path but ignores the 10,000 edge cases hiding in production data. A failed migration does not just waste the project budget — it disrupts operations, corrupts downstream reporting, and in regulated industries, triggers compliance violations that carry six-figure penalties.

The most common failure point is not the migration itself — it is the source data. Companies that have been running the same database for 10, 15, or 20 years accumulate data quality problems that nobody sees until migration exposes them. Duplicate customer records with slightly different spellings. Address fields containing phone numbers. Nullable columns that were never supposed to be null but have 40,000 null values because a form validation was added in 2016 and nobody backfilled. Date fields stored as strings in three different formats across three eras of the application. A West Michigan manufacturer we assessed had 2.3 million rows in their primary customer table. After deduplication and cleansing, the true number of unique active customers was 840,000 — the rest were duplicates, test records, orphaned entries, and soft-deleted rows that never got purged.

The second failure point is schema mapping. Source and target databases almost never have matching schemas. A single 'address' field in your legacy system maps to five normalized fields in the target. Your AS/400 stores dates as 6-digit integers (YYMMDD) while PostgreSQL expects ISO 8601 timestamps. Your old system uses a 2-character state code stored in a free-text field, meaning 'MI', 'Mi', 'mi', 'Mich', 'Michigan', and 'MICHIGAN' all exist in production. Without a documented transformation ruleset for every column in every table, the migration will silently corrupt data that looks fine in spot checks but breaks business logic in production.

The third failure point is rollback. Companies that plan only for success are gambling with their entire operation. If the migration fails halfway through a 48-hour cutover window, you need to be able to restore the original system to its pre-migration state within hours, not days. Every migration FreedomDev runs includes a documented rollback procedure that has been tested at least once before the production cutover.

83% of data migration projects exceed budget, timeline, or fail entirely — most due to poor planning, not technical limitations

Legacy databases contain years of accumulated data quality issues: duplicates, format inconsistencies, orphaned records, and schema drift

Schema mapping between source and target systems requires transformation rules for every column — miss one and data silently corrupts

No rollback plan means a failed cutover can leave your business offline for days with no path back to the original system

Downtime during migration directly impacts revenue: manufacturing plants, e-commerce stores, and healthcare systems cannot afford 48-hour outages

Compliance risk in regulated industries — HIPAA, SOX, PCI-DSS — where data integrity failures during migration trigger audit findings and fines

Need Help Implementing This Solution?

Our engineers have built this exact solution for other businesses. Let's discuss your requirements.

Proven implementation methodology
Experienced team — no learning on your dime
Clear timeline and transparent pricing

Before-and-After: Schema Mapping and Data Transformation

99.97%

Data accuracy rate across completed migrations (verified by post-migration reconciliation)

15–30%

Records cleansed or corrected during pre-migration data quality phase

Data loss incidents across 150+ enterprise migration projects

60–70%

Cost savings from cleaning data before migration vs. after

4–8 hrs

Average production downtime window (vs. 48–72 hrs industry average)

30 days

Post-migration monitoring and hypercare included with every project

Facing this exact problem?

We can map out a transition plan tailored to your workflows.

The Transformation

Data Cleansing Before Migration: What Gets Cleaned and Why

FreedomDev treats data cleansing as a prerequisite to migration, not an afterthought. Every project begins with a data quality audit of the source system that produces a quantified report: total record counts per table, duplicate detection rates, null value percentages per column, format consistency scores, referential integrity violations, and orphaned record counts. This audit typically reveals that 15–30% of records in a legacy database have at least one data quality issue that will cause problems in the target system. Cleaning before migration costs 60–70% less than cleaning after, because post-migration cleanup means finding and fixing bad data that has already been loaded into a production system where it is being referenced by application code, reports, and downstream integrations.

Our cleansing process covers six categories. Deduplication uses fuzzy matching algorithms (Jaro-Winkler, Levenshtein distance, phonetic matching) to identify duplicate records that simple exact-match queries miss — the kind where 'Johnson Manufacturing LLC', 'Johnson Mfg.', and 'Johnson Mfg LLC' are the same company. Format standardization normalizes dates, phone numbers, addresses, currency values, and coded fields to the target system's format. Referential integrity repair fixes broken foreign key relationships — orders pointing to deleted customers, line items referencing discontinued products. Null value resolution applies business rules to populate required fields that are empty in the source: default values, derived values from other fields, or flagged for manual review. Orphan record cleanup removes or archives records that have no parent and no business purpose. Data type conversion maps source types to target types — converting varchar dates to proper datetime columns, numeric strings to integers, and encoded values to lookup table references.

For companies undergoing legacy modernization, the migration is the single best opportunity to fix data quality problems that have been accumulating for years. A clean migration to a well-designed target schema eliminates technical debt in the data layer that would otherwise persist indefinitely. FreedomDev's database services team designs the target schema in parallel with the cleansing phase, ensuring that the destination database enforces constraints and validations that the legacy system never had.

Source System Data Quality Audit

Before writing a single migration script, we profile every table and column in your source database. Record counts, duplicate rates, null percentages, format consistency, referential integrity violations, and data type mismatches — all quantified in a report that tells you exactly what needs to be cleaned and what it will cost to clean it. Typical audit duration: 3–5 days for databases under 50 tables, 1–2 weeks for 100+ table schemas.

Fuzzy Deduplication Engine

Exact-match deduplication catches obvious duplicates. Our fuzzy matching engine catches the rest: misspellings, abbreviations, name variations, and records that were entered by different people at different times with slightly different formatting. We use Jaro-Winkler similarity scoring, Levenshtein distance, Soundex phonetic matching, and domain-specific rules (e.g., 'LLC' equals 'L.L.C.' equals blank for matching purposes). A typical legacy database yields a 10–25% deduplication rate after fuzzy matching.

Schema Mapping & Transformation Rules

We document every column-to-column mapping between source and target schemas with explicit transformation rules: data type conversions, format changes, value translations (code tables), concatenation or splitting of compound fields, default values for new required columns, and conditional logic for fields that map differently based on record type. This document becomes the single source of truth for the entire migration and is version-controlled alongside the migration scripts.

ETL Pipeline Development

Migration scripts are built as repeatable ETL pipelines, not one-time throwaway scripts. We use SSIS for SQL Server environments, custom Python (pandas, SQLAlchemy) for cross-platform migrations, pgloader for PostgreSQL targets, AWS DMS for cloud migrations, and Apache NiFi or Talend for complex multi-source transformations. Every pipeline is idempotent — it can be re-run safely without creating duplicates or corrupting previously migrated data.

Incremental & Delta Migration

For systems that cannot afford extended downtime, we migrate in phases: initial bulk load of historical data (which can run while the legacy system is still live), followed by delta migrations that capture changes made after the initial load, followed by a final cutover delta that captures the last few hours of changes. This approach reduces the actual downtime window from days to hours or even minutes.

Data Validation & Reconciliation

After every migration run, automated validation scripts compare source and target: row counts per table, checksum comparisons on key columns, referential integrity verification in the target, and business rule validation (e.g., every order has at least one line item, every customer has a valid state code). Discrepancies are flagged in a reconciliation report with root cause analysis before any migration is considered complete.

Want a Custom Implementation Plan?

We'll map your requirements to a concrete plan with phases, milestones, and a realistic budget.

Detailed scope document you can share with stakeholders
Phased approach — start small, scale as you see results
No surprises — fixed-price or transparent hourly

“

We had 15 years of data in an AS/400 system that three other companies told us was too messy to migrate. FreedomDev audited the source, cleaned 340,000 duplicate records, mapped every field to our new PostgreSQL schema, and migrated 8.2 million rows over a weekend with zero data loss. Our team came in Monday morning and everything just worked.

Operations Director—West Michigan Manufacturing Company

Our Process

Discovery & Source System Assessment (1–2 Weeks)

We connect to your source database (SQL Server, Oracle, MySQL, PostgreSQL, AS/400, Access, FoxPro, Progress, or flat files) and run a comprehensive data quality audit. Output: table-by-table profiling report showing record counts, duplicate rates, null percentages, format inconsistencies, referential integrity violations, and estimated cleansing effort. We also document the source schema, identify undocumented business rules embedded in the data, and catalog any stored procedures, triggers, or application-level logic that affects data integrity.

Schema Mapping & Transformation Design (1–2 Weeks)

With the source profiled and the target schema defined (or designed in collaboration with our database services team), we build the complete column-to-column mapping document. Every field gets a documented transformation rule. We identify fields that require manual business decisions — cases where data does not map cleanly and someone who understands the business needs to define the rule. This phase produces the migration specification that both FreedomDev and your team sign off on before any code is written.

Data Cleansing Execution (1–3 Weeks)

Cleansing runs against a copy of the source data, never against production. Deduplication, format standardization, null resolution, orphan cleanup, and referential integrity repair are applied in sequence. Each cleansing step produces a before/after report showing exactly what changed and why. Records that cannot be automatically cleaned are flagged for manual review by your team. Typical cleansing phase touches 15–30% of total records and resolves 90–95% of data quality issues identified in the audit.

ETL Pipeline Development & Testing (2–4 Weeks)

We build the migration pipeline as a repeatable, idempotent process. Development happens against a staging copy of the cleansed source data and a staging instance of the target database. Testing covers full migration runs, delta migration runs, error handling (what happens when a record fails mid-batch), performance testing at production data volumes, and rollback procedure verification. Every test run produces a reconciliation report comparing source and target.

Rehearsal Migration (3–5 Days)

A full dress rehearsal of the production migration, run on the most current copy of source data available. This rehearsal validates the entire end-to-end process: cleansing, transformation, loading, validation, and reconciliation. It also benchmarks actual migration duration so we can plan the production cutover window with confidence. We run the rollback procedure during rehearsal to verify it works. Any issues discovered trigger a fix-and-re-rehearse cycle — we do not proceed to production until the rehearsal completes cleanly.

Production Cutover & Validation (1–3 Days)

The production cutover follows a minute-by-minute runbook developed during rehearsal. Source system freeze, final delta extraction, cleansing of delta records, migration execution, validation and reconciliation, application connection switching, smoke testing by your team, and formal go/no-go decision. If validation fails, we execute the tested rollback procedure and revert to the source system. Post-cutover, we monitor the target system for 30 days to catch any data issues that surface during real-world use.

Before vs After

Metric	With FreedomDev	Without
Data Quality Audit	Full profiling of every table and column with quantified quality scores	DIY/automated tools: no source analysis — you migrate dirty data as-is
Data Cleansing	Fuzzy deduplication, format standardization, null resolution, integrity repair	Automated tools: basic dedup at best — no business-rule-aware cleansing
Schema Mapping	Documented column-to-column mappings with transformation rules, reviewed and signed off	DIY: ad hoc field mapping discovered during migration — gaps found in production
Legacy System Support	SQL Server, Oracle, AS/400, Access, FoxPro, Progress, flat files, COBOL data stores	AWS DMS / Azure DMS: limited to supported database engines — no flat files, no AS/400 RPG
Rollback Planning	Tested rollback procedure rehearsed before every production cutover	DIY: hope the backup works — rollback never tested until you need it
Downtime Window	4–8 hours typical (incremental + delta migration strategy)	DIY/automated tools: 24–72 hours for full dump-and-load approach
Post-Migration Validation	Automated reconciliation: row counts, checksums, referential integrity, business rules	DIY: manual spot checks on a handful of records
Ongoing Support	30-day hypercare + optional long-term data quality monitoring	Automated tools: migration complete, you are on your own

Ready to Solve This?

Schedule a direct technical consultation with our senior architects.

Explore More

Database Services Systems Integration Custom Software Development Manufacturing Healthcare Distribution Finance

Frequently Asked Questions

How long does a data migration project take?

Timeline depends on three factors: the size and complexity of the source database, the data quality condition, and the downtime tolerance. A straightforward migration between two modern databases with clean data and compatible schemas — for example, moving from MySQL to PostgreSQL with 50 tables and under 10 million rows — takes 4–6 weeks from kickoff to production cutover. That includes 1 week for discovery and profiling, 1 week for schema mapping, 1 week for cleansing, 1–2 weeks for pipeline development and testing, and a rehearsal-plus-cutover window. Complex migrations involving legacy systems (AS/400, FoxPro, Access), significant data quality issues requiring extensive cleansing, or multi-source consolidation (merging data from 3–5 separate databases into one target) run 8–16 weeks. The longest projects we run are ERP migrations for manufacturers with 20+ years of data in systems like Epicor, MAPICS, or SAP — those can take 3–6 months because the source data contains decades of accumulated business logic that has to be understood, documented, and correctly translated to the new schema. We publish a detailed timeline during the discovery phase so there are no surprises. Every timeline includes buffer for the cleansing phase, because data quality issues always take longer to resolve than initial estimates suggest.

What happens to my data during migration?

Your production data is never touched until the final cutover. All cleansing, transformation, and testing work happens on copies. Here is the sequence: we extract a full copy of your source database to our staging environment. All profiling, cleansing, and transformation work runs against this copy. We build and test the migration pipeline against staging copies of both source and target. When we run the rehearsal migration, we use the most current copy available. Only during the final production cutover do we touch the live source system, and even then, the first step is a verified backup. During the actual migration, your data moves through four stages: extraction from the source (read-only against source), cleansing and transformation in a staging layer (no connection to either production system), loading into the target database (write operations against target only), and validation and reconciliation (read-only comparison of source and target). At no point is your source data modified in place. If the migration fails or validation reveals issues, we execute the rollback procedure and your original system is restored from the verified backup taken before cutover. We maintain backups of the source data, the intermediate staging data, and the target data at each stage of the process so we can trace any discrepancy back to its origin.

Can you migrate data from legacy databases?

Yes — legacy database migration is the core of what FreedomDev does and the area where we have the most experience. We have migrated data from SQL Server (all versions back to SQL Server 2000), Oracle (9i through 19c), PostgreSQL, MySQL, IBM AS/400 (DB2 for i), Microsoft Access (97 through current), FoxPro, Progress OpenEdge, Informix, Sybase, dBASE, FileMaker, and flat file formats including CSV, fixed-width, EDI, XML, and COBOL copybook-defined data files. For AS/400 systems specifically, we work with RPG programs, physical files, logical files, and data queues — extracting data either through ODBC connections, file transfer via FTP, or direct SQL against DB2 for i. For Access databases, we handle the particular challenges that Access migrations present: memo fields with inconsistent encoding, linked tables pointing to network shares that no longer exist, and VBA-embedded business logic that affects data integrity. The key to legacy migration is understanding not just the data but the business rules embedded in the application layer. A 20-year-old ERP system has decades of logic in stored procedures, triggers, and application code that silently shapes the data. We reverse-engineer those rules during discovery so the migration preserves the implicit constraints that the legacy system enforced but never formally documented.

How do you handle data cleansing during migration?

Data cleansing runs as a distinct phase between source extraction and target loading — never skipped, never combined with the migration itself. The process begins with the data quality audit, which quantifies every issue in the source: duplicate record rates, null value percentages per column, format inconsistencies, referential integrity violations, and data type mismatches. This audit typically reveals that 15–30% of records in a legacy database have at least one issue that will cause problems in the target. Cleansing then proceeds through six categories in order. First, deduplication: we run fuzzy matching algorithms (Jaro-Winkler, Levenshtein, Soundex) against entity tables — customers, vendors, contacts, products — to identify records that represent the same real-world entity but differ in spelling, formatting, or abbreviation. A typical legacy system yields 10–25% duplicate rates after fuzzy matching. Merge rules are defined with your team (which record is the master, how to combine data from duplicates). Second, format standardization: dates, phone numbers, addresses, postal codes, state codes, and currency values are normalized to the target system's format. Third, referential integrity repair: broken foreign keys are either repaired (reassigned to the correct parent record) or flagged for review. Fourth, null resolution: required fields that are null in the source get default values, derived values, or manual review flags. Fifth, orphan cleanup: records with no parent and no business purpose are archived. Sixth, data type conversion: varchar dates become datetime, numeric strings become integers, and encoded values become lookup references. Every cleansing step produces a log showing exactly which records were modified and what changed, giving you a complete audit trail.

What is the risk of data loss during migration?

The risk of data loss during a properly executed migration is near zero, but it is never actually zero — which is why every precaution matters. Industry data from Bloor Research shows that 30–40% of data migration projects experience some form of data loss or corruption, almost always due to inadequate testing, missing transformation rules, or absent rollback procedures. FreedomDev has completed over 150 enterprise migration projects with zero data loss incidents, and that record exists because of specific practices, not luck. First, we never modify source data in place — all work happens on copies, and the source system is backed up and verified before cutover. Second, every migration pipeline is idempotent and includes transaction-level error handling: if a batch fails, it rolls back cleanly without partial writes. Third, post-migration validation is automated and exhaustive — row counts per table, column-level checksums, referential integrity verification, and business rule validation (e.g., every order must have line items, every employee must have a department). Fourth, the rehearsal migration catches issues before production — we run the entire process end-to-end on current data and verify the results before touching the live system. Fifth, the rollback procedure is tested during rehearsal so we know it works before we need it. The largest risk factor in any migration is not the tools or the process — it is undocumented business logic in the source system that creates implicit data relationships not captured in the schema. Our discovery phase specifically targets this risk by interviewing users, analyzing application code, and running data pattern analysis to surface hidden rules. If you want to reduce your migration risk to the lowest possible level, invest time in the discovery and cleansing phases. Every dollar spent on pre-migration preparation saves $5–$10 in post-migration firefighting.

Stop Working For Your Software

Make your software work for you. Let's build a sensible solution.