We’ve done data integration a lot, so we know the options and how each perform. We’ll implement the right data transformations for you, your systems, and your database of the future. We’ll deal with change data capture issues, data quality, metadata management, incremental load processing, and so much more. We’ll put it into production and make it easy to administer, so it’s not the headache it once was.
We’ve got a number of technologies we like to use like Informatica Powercenter, IBM DataStage, SAP BODS etc. Some of the open source projects, like Pentaho Data Integration, can get you a lot of value for a small investment. All of them have flexibility to do whatever model best works for the kind of data you are collecting, and the kind of place you’re storing it in.
Keeping the Elephant in the Room
Part of modern ETL is recognizing and being able to take advantage of modern data platforms, like Hadoop, MongoDB, Cassandra, and others. Not every next generation platform solves everyone’s needs, but your data management layer sure needs to deal with it. When we do ETL projects, we’re sure to pick the right technology to do not just one flavor of processing, but many flavors. Because the number of data sources in your company sure isn’t decreasing.