Renungan Kristen - Kendali Hidup #Sharing

VERSI PDF

Apa yang mengendalikan hidupmu?
Ada banyak hal yang yang mengedalikan hidup kita, seperti : keadaan, nilai-nilai, dan emosi.
Ada 5 hal yang paling umum :

1. Banyak orang dikendalikan rasa bersalah. Orang-orang yang dikendalikan rasa bersalah dimanipulasi oleh ingatan masa lalu dan membiarkan masa lalu mengendalikan masa depan mereka. Maksud dan tujuan Tuhan tidak dibatasi masa lalu kita. Dia mengubah seorang pembunuh bernama Musa menjadi seorang pemimpin dan seorang pengecut bernama Gideon menjadi seorang pahlawan pemberani, dan Dia (Tuhan) berkuasa dan mampu melakukan perkara-perkara ajaib dengan sisa hidup kita.
2. Banyak orang dikendalikan dendam dan kemarahan. Mereka terus berpegang pada luka batin dan tidak pernah melupakannya. Bukannya melepaskan penderitaan mereka melalui pengampunan, mereka mengorek-ngorek luka itu lagi dan lagi di benak mereka. Dendam selalu menyakiti kita lebih dari orang yang menyakiti kita. Mereka yang menyakiti kita di masa lalu tidak dapat terus menyakiti kita kecuali kita terus berpegang pada luka batin itu melalui dendam. Masa lalu adalah masa lalu! Tidak ada yang mampu mengubahnya. Belajarlah dari masa lalu, dan lepaskan. 
3. Banyak orang dikendalikan rasa takut. Orang – orang yang dikendalikan rasa takut sering kehilangan kesempatan-kesempatan besar karena mereka takut bertualang. Kita harus melawan rasa takut kita dengan senjata iman dan kasih.
4. Banyak orang dikendalikan materialisme. Hal ini mengendalikan orang untuk selalu menginginkan lebih berdasarkan kesalahpahaman bahwa memiliki lebih akan membuat lebih bahagia, lebih penting, dan lebih aman, tapi semua itu tidak benar. Harta hanya menyediakan kebahagiaan sesaat. Kekayaan dapat hilang dalam sekejap melalui beragam faktor yang tak terkendali. Keamanan yang sesungguhnya hanya dapat ditemukan dalam sesuatu yang tidak pernah bisa diambil dari kita, yaitu hubungan kita dengan Tuhan.
5. Banyak orang dikendalikan oleh kebutuhan akan persetujuan. Mereka mengijinkan ekspektasi-ekspektasi orang tua atau pasangan atau anak atau guru atau teman mengendalikan hidup mereka.

Cara mengendalikan hidup kita.
1. Berpikir Sederhana 

Roma 12:16
Hendaklah kamu sehati sepikir dalam hidupmu bersama; janganlah kamu memikirkan perkara-perkara yang tinggi, tetapi arahkan dirimu kepada perkara-perkara yang sederhana. Janganlah menganggap dirimu pandai! 

Terkadang kita berpikir terlalu jauh sehingga tidak menghasilkan apa-apa. Seringkali kita berpikir terlalu rumit sampai-sampai kita sendiri tidak tahu bagaimana melaksanakannya.

Roma 12:3 

Berdasarkan kasih karunia yang diianugerahkan kepadaku, aku berkata kepada setiap orang di antara kamu : Janganlah kamu memikirkan hal-hal yang lebih tinggi dari pada yang patut kamu pikirkan, tetapi hendaklah kamu berpikir begitu rupa, sehingga kamu menguasai diri menurut ukuran iman, yang dikaruniakan Allah kepada kamu masing-masing.

  
Tuhan sudah memberikan kita kemampuan dan kepandaian, hanya tinggal terserah kita, apakah kita mau mengupayakannya atau tidak. Tapi percayalah bahwa kita kan berhasil asal kita mau memulainya dari diri kita, hari ini juga, dan bahkan sekarang juga. 

2. Mengatasi Perasaan Bersalah
Setiap orang telah berdosa dan salah satu akibat dosa adalah rasa bersalah. Kita bisa bersyukur untuk rasa bersalah karena hal itu mendorong kita untuk mendapatkan pengampunan. Saat seseorang berbalik dari dosa kepada Yesus Kristus dalam iman, dosanya diampuni. Daftar beberapa perbuatan tercela yang sudah diampuni ada di 1 Korintus 6:9-11.

Namun demikian, kebebasan dari dosa tidak selalu berarti kebebasan dari perasaan bersalah. Sekalipun dosa sudah diampuni, kita masih mengingatnya.  Ketika seorang Kristen mengalami perasaan bersalah, dia harus melakukan hal-hal berikut ini :

• Akui semua dosa yang diketahui yang sebelumnya belum diakui. (Mazmur 32:3-5)
• Minta Tuhan mengungkapkan dosa apa saja yang masih perlu diakui. Beranilah untuk bersikap sama sekali terbuka dan jujur di hadapan Allah. (Mazmur 139:23-24)
• Percaya pada janji Allah bahwa Dia akan mengampuni dosa dan menyingkirkan kesalahan berdasarkan darah Kristus. (1 Yohanes 1:9 ; Mazmur 85:2 ; Mazmur 86:5 ; Roma 8:1-2)
• Ketika perasaan bersalah muncul untuk dosa-dosa yang telah diakui dan diampuni, tolak perasaan-perasaan semacam itu sebagai rasa bersalah yang keliru. Allah berpegang pada janji-Nya untuk mengampuni. (Mazmur 103:8-12) 
• Minta pada Tuhan untuk menegur Iblis pendakwa Anda, dan mintalah Tuhan untuk memulihkan sukacita yang datang bersama dengan kebebasan dari rasa bersalah.

3. Mengendalikan Amarah
Kemarahan adalah emosi yang juga diberikan oleh Tuhan. Kemarahan itu seperti sakit kepala; itu adalah sebuah sinyal adanya sesuatu yang salah dan membutuhkan perhatian. 

Mazmur 37:8
Berhentilah marah dan tinggalkanlah panas hati itu, jangan marah, itu hanya membawa kepada kejahatan.

Berikut adalah 5 tahap yang dapat kita lakukan untuk mengendalikan marah versi Buku Anger – Mengatasi Amarah Dengan Cara Yang Sehat (Gary Chapman)

• Secara sadar mengakui kepada diri Anda sendiri bahwa Anda marah. Ucapkan dengan bersuara, ”Aku marah tentang hal ini! Kini apa yang harus aku lakukan?” Pernyataan seperti itu membuat Anda sadar akan amarah Anda sendiri dan juga menolong Anda mengenali baik amarah Anda dan tindakan yang akan Anda ambil. Anda telah menyiapkan tempat untuk menerapkan nalar terhadap amarah Anda.
• Mengendalikan respons langsung Anda. Hindari respons-respons umum tetapi merusak, seperti melampiaskan amarah secara verbal atau fisik, menarik diri, dan berdiam diri. Tolaklah mengambil tindakan yang biasanya Anda ambil saat merasa marah. Menunggu bisa menolong Anda untuk menghindari hal-hal yang mungkin tidak Anda maksudkan dan akan Anda sesali kemudian.
• Carilah fokus amarah Anda. Kata-kata atau tindakan mana oleh orang lain yang membuat Anda marah? Jika orang itu benar-benar bersalah kepada Anda, identifikasi dosa orang itu. Bagaimana ia bersalah kepada Anda? Lalu tentukan seberapa seriusnya pelanggarannya. Beberapa kesalahan bersifat minor dan beberapa sifatnya mayor. Mengetahui keseriusan masalah tersebut seharusnya mempengaruhi respons Anda. 
• Analisa pilihan-pilihan Anda. Bertanyalah kepada diri Anda sendiri, "Apakah tindakan yang sedang kupertimbangkan ini akan memiliki potensi untuk berurusan dengan yang salah dan menolong relasi yang terganggu? Apakah itu yang terbaik bagi orang yang kepadanya aku marah?" Dua pilihan yang paling membangun adalah mengonfrontasi orang itu dengan cara yang menolong atau secara sadar memutuskan untuk mengabaikan masalahnya.
• Mengambil tindakan yang membangun. Jika Anda memilih untuk ”merelakan pelanggaran itu”, maka akuilah amarah Anda dalam doa dan kerelaan Anda untuk menyerahkan orang itu kepada Allah. Lalu lepaskan amarah Anda kepadaNya. Jika Anda memilih untuk mengonfrontasi orang yang telah bersalah kepada Anda, lakukan dengan lemah lembut. Dengarkan penjelasan yang diberikan, itu bisa memberi Anda sudut pandang yang berbeda akan tindakan dan maksud orang itu. Jika orang itu mengakui apa yang dilakukannya salah dan meminta Anda memaafkan, lakukan saja demikian.

4. Mengatasi Kekuatiran dan Ketakutan 
Kekuatiran adalah sebuah perasaan gelisah, ketakutan terhadap sesuatu yang belum terjadi. Perasaan-perasaan ini biasanya terkait dengan pikiran-pikiran negatif atas sesuatu yang mungkin terjadi di masa depan. 

Matius 6:25
Karena itu Aku berkata kepadamu: Janganlah kuatir akan hidupmu, akan apa yang hendak kamu makan atau minum, dan janganlah kuatir pula akan tubuhmu, akan apa yang hendak kamu pakai. Bukankah hidup itu lebih penting dari pada makanan dan tubuh itu lebih penting dari pada pakaian?

Dalam kehidupan ini seringkali kita takut akan hal-hal yang “belum tentu” terjadi. Tidak jarang kita kuatir terhadap masa depan kita, masa depan anak kita, pekerjaan kita dan sebagainya.

Tidak seorang pun yang hidup tanpa kekuatiran, tak satupun kebal dari kekuatiran. Jika seseorang berkata bahwa dia tidak peduli akan apapun di dunia ini, maka dia ada dalam penyangkalan.

Apa  yang harus kita perbuat ketika rasa kuatir menyerang pikiran kita?

• Rasul Paulus menasehati, “… nyatakanlah dalam segala hal keinginanmu kepada Allah dalam doa dan permohonan dengan ucapan syukur. Damai sejahtera Allah, yang melampauai akal, akan memelihara hati dan pikiranmu dalam Kristus Yesus (Filipi 4:6-7)
• Rasul Petrus menasehati, “Serahkanlah segala kekuatiranmu kepada-Nya, sebab Ia memelihara kamu. (1 Petrus 5:7)

5. Jangan Serakah, Jangan Menjadi Hamba Uang
Apa itu serakah? Menginginkan, bahkan merampas sesuatu yang Tuhan karuniakan kepada orang lain.

Membandingkan diri dengan orang lain itu biasa, tapi itu adalah sifat dosa yang harus kita pertanggungjawabkan pada Tuhan. Merasa puas atas apa yang sudah Tuhan beri, tidak banyak membanding-bandingkan diri dengan orang adalah dasar dari kerohanian yang stabil. Kadang perasaan susah, tidak enak muncul, karena kita membanding-bandingkan diri dengan orang lain: Mengapa ia begini, mengapa saya begitu? Kemudian disusul dengan rasa tidak puas akan apa yang sudah Tuhan berikan pada kita.

Batasan serakah :

• Kalau kekayaan kita peroleh dengan jalur yang benar, tentu tidak bisa disebut serakah. 
• Kalau kekayaan yang kita simpan adalah hasil dari perjuangan atau bijaksana kita, tentu tidak bisa disebut serakah.
• Kalau kita menggunakan harta kita sejalan dengan prinsip Tuhan, dengan pimpinan Roh Kudus, bukan dengan egois, itu juga tidak bisa disebut budak harta.

Janganlah kita menjadi budak dosa, budak harta, budak nafsu diri kita sendiri. Apa maksudnya? Jangan sampai hidup kita berantakan, karena kita tak mampu mengendalikan nafsu, maka nafsulah yang akan mengendalikan kita. Karena kita tak mampu mengendalikan uang, maka uanglah yang akan mengendalikan kita, dan kita menjadi hamba uang.

Apa Artinya Menjadi Hamba Uang?
Menjadi hamba uang artinya adalah menganggap uang sebagai yang terutama dalam hidup ini, dan menjadikan uang sebagai segala-galanya. Menjadi hamba uang berarti menganggap bahwa segala sesuatunya dapat dibeli dengan uang, dan semua yang dilakukan haruslah diukur dengan uang.

• Uang memang bisa membeli buku, tetapi tidak bisa membeli hikmat. 
• Uang memang bisa membeli pengawal, tetapi tidak bisa membeli keamanan.
• Uang memang bisa membeli salib, tetapi tidak bisa membeli keselamatan.
• Uang memang bisa membeli cincin kawin, tetapi tidak bisa membeli cinta.
• Uang dapat membeli kasur yang empuk, tetapi tidak bias membeli tidur yang nyenyak.
• Banyak hal yang dapat dibeli dengan uang, tetapi uang tidak dapat membeli semua hal.

Caranya sebetulnya mudah, jadilah hamba Tuhan dan bukan hamba uang. Jadikan Tuhan sebagai yang terutama dalam hidup kita, dan bukan uang yang terutama. Jika kita menjadikan Tuhan sebagai tuan kita, maka kita menjadi hamba Tuhan. Jika kita menjadikan uang sebagai tuan kita, maka kita manjadi hamba uang. Kita harus memilih salah satu, karena tidak ada orang yang dapat mengabdi ke dua tuan. 

Matius 6:24
Tak seorang pun dapat mengabdi kepada dua tuan. Karena jika demikian, ia akan membenci yang seorang dan mengasihi yang lain, atau ia akan setia kepada yang seorang dan tidak mengindahkan yang lain. Kamu tidak dapat mengabdi kepada Allah dan kepada Mamon.

1 Timotius 6:10

Karena akar segala kejahatan ialah cinta uang. Sebab oleh memburu uanglah beberapa orang telah menyimpang dari iman dan menyiksa dirinya dengan berbagai-bagai duka.

Sesungguhnya kita salah ketika kita mencari uang tapi tidak mencari Tuhan. Pastikanlah bahwa kita harus mencari Tuhan terlebih dahulu, maka selanjutnya uang akan datang dengan sendirinya kepada kita. 

Matius 6:33
Tetapi carilah dahulu Kerajaan Allah dan kebenarannya, maka semuanya itu akan ditambahkan kepadamu.

6. Manajemen Ekspektasi
Hidup tidak selalu berjalan seperti yang kita mau. Mungkin kita tidak dapat promosi, pernikahan yang tidak berjalan seperti yang sebelumnya dibayangkan, atau suatu musibah terjadi pada seseorang yang kita kasihi atau yang dekat dengan kita.

Salah satu hal yang sulit kita lakukan sebagai orang Kristen adalah menghadapi ekspektasi yang tidak terpenuhi. Kita memiliki kecenderungan untuk menulis rencana kehidupan kita sendiri dan kemudian mengharapkan Allah membuat semuanya terjadi. Misalnya, kita memiliki harapan bahwa jika kita rajin belajar di sekolah dan mendapatkan pendidikan yang baik, kita akan bisa mendapatkan pekerjaan yang baik, mendapatkan uang baik, dan kemudian memiliki kehidupan yang baik. Tapi hidup terkadang memiliki rencana sendiri.

Alkitab memiliki banyak contoh bagaimana orang Kristen seringkali memiliki ekspektasi yang tidak terpenuhi, contoh :

• Ayub - Meskipun Ayub mengabdi kepada Allah, Ayub kehilangan rumahnya, keluarganya, harta miliknya, dan bahkan kesehatannya. Namun pada akhirnya, Tuhan memberkati kesetiaan Ayub dengan cara yang tidak pernah terbayangkan.
• Musa - Seorang pemuda yang dipanggil Allah untuk membebaskan umat-nya dari Mesir. Dalam prosesnya, Musa ternyata mengalami banyak tantangan dan hambatan: mulai dari umat Israel yang tidak percaya padanya, bersungut-sungut dan lemah imannya.

Kita cenderung terperangkap dalam sesuatu yang terjadi pada kita, tapi :

• Jangan biarkan “Mengapa” membayangi “Siapa”

Ibrani 10:25

Janganlah kita menjauhkan diri dari pertemuan-pertemuan ibadah kita, seperti dibiasakan oleh beberapa orang, tetapi mariklah kita saling menasehati, dan semakin giat melakukannya menjelang hari Tuhan yang mendekat.

• Latih mata kita untuk melihat dan telinga untuk mendengar

Matius 13:15-16

Sebab hati bangsa ini telah menebal, dan telinganya berat mendengar, dan matanya melekat tertutup; supaya jangan mereka melihat dengan matanya dan mendengar dengan telinganya dan mengerti dengan hatinya, lalu berbalik sehingga Aku menyembuhkan mereka. Tetapi berbahagialah matamu karena melihat dan telingamu karena mendengar.

Ibarat kita sedang bermain alat musik, ketika kita bermain sendiri, kedengarannya baik-baik saja. Tapi jika kita bermain alat musik tersebut pada waktu yang bersamaan dengan Allah, kita akan mendapat perbedaan antara kita dengan Allah.
Ketika ketidaksesuaian terjadi, hendaklah kita mengsinkronkan kembali diri kita dengan Allah.

• Masa depanmu tidak ditetapkan oleh apa yang telah terjadi tetapi oleh apa yang akan terjadi

Yesaya 40:31
tetapi orang-orang yang menanti-nantikan Tuhan mendapat kekuatan baru : mereka seumpama rajawali yang naik terbang dengan kekuatan sayapnya; mereka berlari dan tidak menjadi lesu, mereka berjalan dan tidak menjadi lelah.

Isaiah 40:31 (NKJV)
but those who wait on the Lord shall renew their strength; they shall mount up with wings like eagles, they shall run and not be weary, they shall walk and not faint.

Kata  "menanti-nantikan" atau “menunggu (wait) menunjukkan sebuah kesungguhan bahwa Allah dapat dan akan melakukan sesuatu dengan situasi ini.

7. Percayalah Harapan Tetap Ada

Roma 12:11-12
Janganlah hendaknya kerajinanmu kendor, biarlah rohmu menyala-nyala dan layanilah Tuhan. Bersukacitalah dalam pengharapan, sabarlah dalam kesesakan, dan bertekunlah dalam doa.

Hidup di dalam Tuhan adalah hidup yang penuh dengan pengharapan. Janganlah biarkan pengharapan yang kita miliki itu menjadi kendor. Bila hal itu terjadi maka iblis akan masuk ke dalam hidup kita dan membuat hidup kita menjadi penh keraguan.

Ada 3 hal yang dapat membuat roh kita tetap menyala – nyala :

• Bersukacitalah dalam pengharapan

Pengharapan itu selalu ada karena Tuhan sendiri yang telah berjanji. 

Titus 2:11-13
Karena kasih karunia Allah yang menyelamatkan semua manusia sudah nyata. Ia mendidik kita supaya kita meninggalkan kefasikan dan keinginan-keingan duniawi dan supaya kita hidup bijaksana, adil dan beribadah di dalam dunia sekarang ini dengan menantikan penggenapan pengharapan kita yang penuh bahagia dan pernyataan kemuliaan Allah yang Mahabesar dan Juruselamat kita Yesus Kristus.

Tuhan akan memberikan lebih dari apa yang kita harapkan danTuhan akan memberikan tepat pada waktunya.

• Sabar dalam kesesakan

Kita harus bisa bersabar dalam segala penderitaan yang terjadi di dalam hidup kita. Penderitaan akan membawa kita melihat mujizat Tuhan. Tuhan akan memberikan pertolongan diwaktu yang tepat karena waktu yang tepat dari Tuhan itu adalah yang terbaik. 

Ayub 36:15
Dengan sengsara Ia menyelamatkan orang sengsara, dengan penindasan Ia membuka telinga mereka.

Kita akan menuai kemuliaan-Nya bila kita mau bersabar.

• Bertekun dalam doa

Roma 8:26
Demikian juga Roh membantu kita dalam kelemahan kita; sebab kita tidak tahu, bagaimana sebenarnya harus berdoa; tetapi Roh sendiri berdoa untuk kita kepada Allah dengan keluhan – keluhan yang tidak terucapkan.

Ada kuasa dalam doa.

Keuntungan dari hidup yang dikendalikan tujuan.
Tidak ada yang lebih penting daripada mengetahui maksud dan tujuan Tuhan bagi hidup kita. Tanpa suatu maksud dan tujuan, hidup seperti gerakan tanpa arti, kegiatan tanpa arah, dan peristiwa tanpa alasan.

1. Mengetahui tujuan memberi makna pada hidupmu. Tanpa Tuhan, hidup tidak memiliki tujuan, dan tanpa tujuan, hidup tidak mempunyai makna. Tanpa makna, hidup tidak mempunyai arti penting atau harapan. Tragedi terbesar bukanlah kematian, tapi hidup tanpa tujuan. Harapan sangat esensial bagi hidup kita seperti udara dan air. Kita membutuhkan harapan untuk bisa mengatasi apapun. Harapan datang dari memiliki suatu tujuan. 
2. Mengetahui tujuan menyederhanakan hidup kita. Hal itu mendefinisikan apa yang kita lakukan dan apa yang tidak kita lakukan. Tanpa tujuan yang jelas kita tidak memiliki landasan untuk keputusan-keputusan, alokasi waktu dan pemanfaatan sumber daya kita.
3. Mengetahui tujuan memfokuskan hidup kita. Ia mengkonsentrasikan usaha dan energi kita pada apa yang penting. Kita menjadi efektif dengan cara menjadi selektif. Tanpa suatu tujuan yang jelas, kita akan terus mengubah arah, pekerjaan, hubungan, gereja atau hal-hal eksternal lain, berharap masing-masing perubahan itu akan menyelesaikan kebingungan dan mengisi kekosongan hati. Jika kita ingin hidup kita mempunyai dampak, fokuskan! Lakukan hanya apa yang paling penting.
4. Mengetahui tujuan memotivasi hidup kita. Tujuan selalu menghasilkan gairah. Tidak ada yang memberi energi seperti suatu tujuan yang jelas. 
5. Mengetahui tujuan menyiapkan hidup kita bagi keabadian. Apa yang paling penting tidak akan berupa apa  yang orang lain katakan tentang hidup kita, tetapi pada apa yang Tuhan katakan. Hidup untuk menciptakan suatu warisan duniawi adalah suatu tujuan yang picik. Pemanfaatan waktu yang lebih bijaksana adalah membangun suatu warisan surgawi. Kita ada di dunia ini untuk menyiapkan diri kita bagi keabadian.

Semoga bisa menjadi berkat dan renungan bagi kehidupan kita semua. Amin.
-Nico-

VERSI PDF

Reporting Vs Analytics

I was asked to create a comparison between reporting and analytics for my customers. In my perspective, they are very different in many ways except they are integrated to each other in terms of data. Both of them created based on data from the database, but cooked in different way using different tools and consumed by different people. I would like to share the summary of them below :

Next, I add an article from a blog by Madish Desai which attract me :

...

I had met up with yet another prospect last week who was not aware of difference between reporting and analytical solutions. I would blame reporting tool vendor for this who has started confusing customers by positioning reporting tools as analytical solution. There are several differences between reporting and analytical solutions.

1. Business Objective
Reporting: Reporting solutions will help you measure performance of various business entities relative to business plan or target. It will help you convert data into information

Analytics: Analytical solutions will help you identify new products, customer segments, reduce cost, risk & fraud. It will help you convert information into knowledge.

Example: Reporting solution will tell you number of stock outs by items by store whereas Analytical solution will tell you about optimum amount of quantity that you need to keep in your store to minimize stock outs and opportunity cost.

2. Information Output
Reporting: Reporting solution output will help you quantify past performance.

Analytics: Analytical solution output will help you infer unknown facts and relationships. It will also help you quantify future probabilities.

Example: Reporting solution will tell you about best selling products in your portfolio whereas analytical solution will tell you about probability of  buying a particular product when your customer visits your store next time.

3. Output 
Reporting: Historical standard reports, dashboards, KPIs, cubes for OLAP.

Analytics: Predictive models, scores, forecasts.

Example

• Reporting: Top 10 products by revenue, Top 10 customers by region
• Analytics: Cross Sell/Up Sell Model, Forecasting by Product by Region by Time

4. Queries
Reporting: Known, simple queries which can be easily optimized.

Analytics: Queries that become very complex as they evolve via iteration.

Reporting solution will help you answer the following questions

1. What happened? When did it happen?
2. How many? How often? Where?
3. Where exactly is the problem? How do I find the answers?

Analytical solution will help you answer the following questions

1. Why is it happening? What opportunities am I missing?
2. What if these trends continue? How much is needed? When will it be needed?
3. What will happen next? How will it affect my business?
4. How do we do things better? What is the best decision for a complex problem?

Thomas Davenport has rightly said "Organizations that fail to invest in the proper analytic technologies will be unable to compete in a fact-based business environment."

Davenport says organizations successfully competing on analytics exhibit a set of common attributes, including:

• CEO commitment – To use analytics as a basis for competition requires commitment from the top of the organization. It requires an allocation of resources, long-term funding and, in some cases, a shift in culture.

• Strategic focus – Successful users of analytics don't just use analytics in general. They first define their distinctive capability and then use analytics to support that capability.

• Enterprise application – Firms that compete on analytics don't manage it locally. They eliminate fiefdoms of data, centralize the data and expertise, and manage analytics at the enterprise level.

...

Source 1 : Phil Simon Blog
Source 2 : Manish Desai Blog

Understanding

Good evening people. In this life, we face a lot of things. Many things cannot be controlled by us, it is given. The other things happen and controlled by us, it is a choice. Let me give you some example of things that cannot be controlled, it is where we were born and where we come from. Next, some example of what we can control, what is our job and what we choose to be. Don't waste your time trying to change something given.

As a human, we learn how to live, we learn how to survive and we learn how to enjoy this life. It is about learning. Why we are learning? Because we do not understand everything about this life, especially things that haven't happened.
 

You will never truly understand something until it actually happens to you.

I think the quote above is very true. It is very hard to understand something that haven't happened. It is extremely hard to imagine something in a whole picture. The sense of happiness, sadness, curiosity, loss, gratitude, etc won't be the same and maybe different when something happens to us directly. After all, it will become history, and we will gain the understanding. I prefer to call it experience. That experience will be something that we can share with others to help them understand the pieces of understanding.

The What, Why, and How of Master Data Management - Part 2 - By Microsoft

Previously : Introduction & Why Should I Manage Master Data?

What Is Master Data Management?

For purposes of this article, we define Master Data Management (MDM) as the technology, tools, and processes required to create and maintain consistent and accurate lists of master data. There are a couple things worth noting in this definition. One is that MDM is not just a technological problem. In many cases, fundamental changes to business process will be required to maintain clean master data, and some of the most difficult MDM issues are more political than technical. The second thing to note is that MDM includes both creating and maintaining master data. Investing a lot of time, money, and effort in creating a clean, consistent set of master data is a wasted effort unless the solution includes tools and processes to keep the master data clean and consistent as it is updated and expanded.

While MDM is most effective when applied to all the master data in an organization, in many cases the risk and expense of an enterprise-wide effort are difficult to justify. It may be easier to start with a few key sources of Master Data and expand the effort, once success has been demonstrated and lessons have been learned. If you do start small, you should include an analysis of all the master data that you might eventually want to include, so you do not make design decisions or tool choices that will force you to start over when you try to incorporate a new data source. For example, if your initial Customer master implementation only includes the 10,000 customers your direct-sales force deals with, you don't want to make design decisions that will preclude adding your 10,000,000 Web customers later.

An MDM project plan will be influenced by requirements, priorities, resource availability, time frame, and the size of the problem. Most MDM projects include at least these phases: 

1) Identify sources of master data. This step is usually a very revealing exercise. Some companies find they have dozens of databases containing customer data that the IT department did not know existed.

2) Identify the producers and consumers of the master data. Which applications produce the master data identified in the first step, and—generally more difficult to determine—which applications use the master data. Depending on the approach you use for maintaining the master data, this step might not be necessary. For example, if all changes are detected and handled at the database level, it probably does not matter where the changes come from.

3) Collect and analyze metadata about for your master data. For all the sources identified in step one, what are the entities and attributes of the data, and what do they mean? This should include attribute name, datatype, allowed values, constraints, default values, dependencies, and who owns the definition and maintenance of the data. The owner is the most important and often the hardest to determine. If you have a repository loaded with all your metadata, this step is an easy one. If you have to start from database tables and source code, this could be a significant effort.

4) Appoint data stewards. These should be the people with the knowledge of the current source data and the ability to determine how to transform the source into the master-data format. In general, stewards should be appointed from the owners of each master-data source, the architects responsible for the MDM systems, and representatives from the business users of the master data.

5) Implement a data-governance program and data-governance council. This group must have the knowledge and authority to make decisions on how the master data is maintained, what it contains, how long it is kept, and how changes are authorized and audited. Hundreds of decisions must be made in the course of a master-data project, and if there is not a well-defined decision-making body and process, the project can fail, because the politics prevent effective decision making.

6) Develop the master-data model. Decide what the master records look like: what attributes are included, what size and datatype they are, what values are allowed, and so forth. This step should also include the mapping between the master-data model and the current data sources. This is normally both the most important and most difficult step in the process. If you try to make everybody happy by including all the source attributes in the master entity, you often end up with master data that is too complex and cumbersome to be useful. For example, if you cannot decide whether weight should be in pounds or kilograms, one approach would be to include both (WeightLb and WeightKg). While this might make people happy, you are wasting megabytes of storage for numbers that can be calculated in microseconds, as well as running the risk of creating inconsistent data (WeightLb = 5 and WeightKg = 5). While this is a pretty trivial example, a bigger issue would be maintaining multiple part numbers for the same part. As in any committee effort, there will be fights and deals resulting in sub-optimal decisions. It's important to work out the decision process, priorities, and final decision maker in advance, to make sure things run smoothly.
 
7) Choose a toolset. You will need to buy or build tools to create the master lists by cleaning, transforming, and merging the source data. You will also need an infrastructure to use and maintain the master list. These functions are covered in detail later in the paper.

You can use a single toolset from a single vendor for all of these functions, or you might want to take a best-of-breed approach. In general, the techniques to clean and merge data are different for different types of data, so there are not a lot of tools that span the whole range of master data.

The two main categories of tools are Customer Data Integration (CDI) tools for creating the customer master and Product Information Management (PIM) tools for creating the product master. Some tools will do both, but generally they are better at one or the other.

The toolset should also have support for finding and fixing data-quality issues and maintaining versions and hierarchies. Versioning is a critical feature, because understanding the history of a master-data record is vital to maintaining its quality and accuracy over time. For example, if a merge tool combines two records for John Smith in Boston, and you decide there really are two different John Smiths in Boston, you need to know what the records looked like before they were merged, in order to "unmerge" them

8) Design the infrastructure. Once you have clean, consistent master data, you will need to expose it to your applications and provide processes to manage and maintain it. This step is a big-enough issue, I devote a section to it later in the document. When this infrastructure is implemented, you will have a number of applications that will depend on it being available, so reliability and scalability are important considerations to include in your design. In most cases, you will have to implement significant parts of the infrastructure yourself, because it will be designed to fit into your current infrastructure, platforms, and applications.

9) Generate and test the master data. This step is where you use the tools you have developed or purchased to merge your source data into your master-data list. This is often an iterative process requiring tinkering with rules and settings to get the matching right. This process also requires a lot of manual inspection to ensure that the results are correct and meet the requirements established for the project. No tool will get the matching done correctly 100 percent of the time, so you will have to weigh the consequences of false matches versus missed matches to determine how to configure the matching tools. False matches can lead to customer dissatisfaction, if bills are inaccurate or the wrong person is arrested. Too many missed matches make the master data less useful, because you are not getting the benefits you invested in MDM to get.

10) Modify the producing and consuming systems. Depending on how your MDM implementation is designed, you might have to change the systems that produce, maintain, or consume master data to work with the new source of master data. If the master data is used in a system separate from the source systems—a data warehouse, for example—the source systems might not have to change. If the source systems are going to use the master data, however, there will likely be changes required. Either the source systems will have to access the new master data or the master data will have to be synchronized with the source systems, so that the source systems have a copy of the cleaned-up master data to use. If it's not possible to change one or more of the source systems, either that source system might not be able to use the master data or the master data will have to be integrated with the source system's database through external processes, such as triggers and SQL commands.

The source systems generating new records should be changed to look up existing master record sets before creating new records or updating existing master records. This ensures that the quality of data being generated upstream is good, so that the MDM can function more efficiently and the application itself manages data quality. MDM should be leveraged not only as a system of record, but also as an application that promotes cleaner and more efficient handling of data across all applications in the enterprise. As part of MDM strategy, all three pillars of data management need to be looked into: data origination, data management, and data consumption. It is not possible to have a robust enterprise-level MDM strategy if any one of these aspects is ignored.

11) Implement the maintenance processes. As we stated earlier, any MDM implementation must incorporate tools, processes, and people to maintain the quality of the data. All data must have a data steward who is responsible for ensuring the quality of the master data. The data steward is normally a business person who has knowledge of the data, can recognize incorrect data, and has the knowledge and authority to correct the issues. The MDM infrastructure should include tools that help the data steward recognize issues and simplify corrections. A good data-stewardship tool should point out questionable matches that were made—customers with different names and customer numbers that live at the same address, for example. The steward might also want to review items that were added as new, because the match criteria were close but below the threshold. It is important for the data steward to see the history of changes made to the data by the MDM systems, to isolate the source of errors and undo incorrect changes. Maintenance also includes the processes to pull changes and additions into the MDM system, and to distribute the cleansed data to the required places.

...

As you can see, MDM is a complex process that can go on for a long time. Like most things in software, the key to success is to implement MDM incrementally, so that the business realizes a series of short-term benefits while the complete project is a long-term process. No MDM project can be successful without the support and participation of the business users. IT professionals do not have the domain knowledge to create and maintain high-quality master data. Any MDM project that does not include changes to the processes that create, maintain, and validate master data is likely to fail. The rest of this paper will cover the details of the technology and processes for creating and maintaining master data. 

#How Do I Create a Master List?

Whether you buy a tool or decide to roll your own, there are two basic steps to creating master data: clean and standardize the data, and match data from all the sources to consolidate duplicates. Before you can start cleaning and normalizing your data, you must understand the data model for the master data. As part of the modeling process, the contents of each attribute were defined, and a mapping was defined from each source system to the master-data model. This information is used to define the transformations necessary to clean your source data.

Cleaning the data and transforming it into the master data model is very similar to the Extract, Transform, and Load (ETL) processes used to populate a data warehouse. If you already have ETL tools and transformation defined, it might be easier just to modify these as required for the master data, instead of learning a new tool. Here are some typical data-cleansing functions: 

• Normalize data formats. Make all the phone numbers look the same, transform addresses (and so on) to a common format.
• Replace missing values. Insert defaults, look up ZIP codes from the address, look up the Dun & Bradstreet number.
• Standardize values. Convert all measurements to metric, convert prices to a common currency, change part numbers to an industry standard.
• Map attributes. Parse the first name and last name out of a contact-name field, move Part# and partno to the PartNumber field.

Most tools will cleanse the data that they can, and put the rest into an error table for hand processing. Depending on how the matching tool works, the cleansed data will be put into a master table or a series of staging tables. As each source is cleansed, the output should be examined to ensure the cleansing process is working correctly.

Matching master-data records to eliminate duplicates is both the hardest and most important step in creating master data. False matches can actually lose data (two Acme Corporations become one, for example) and missed matches reduce the value of maintaining a common list. The matching accuracy of MDM tools is one of the most important purchase criteria. Some matches are pretty trivial to do. If you have Social Security numbers for all your customers, or if all your products use a common numbering scheme, a database JOIN will find most of the matches. This hardly ever happens in the real world, however, so matching algorithms are normally very complex and sophisticated. Customers can be matched on name, maiden name, nickname, address, phone number, credit-card number, and so on, while products are matched on name, description, part number, specifications, and price. The more attribute matches and the closer the match, the higher degree of confidence the MDM system has in the match. This confidence factor is computed for each match, and if it surpasses a threshold, the records match. The threshold is normally adjusted depending on the consequences of a false match. For example, you might specify that if the confidence level is over 95 percent, the records are merged automatically, and if the confidence is between 80 percent and 95 percent, a data steward should approve the match before they are merged.

Most merge tools merge one set of input into the master list, so the best procedure is to start the list with the data in which you have the most confidence, and then merge the other sources in one at a time. If you have a lot of data and a lot of problems with it, this process can take a long time. You might want to start with the data from which you expect to get the most benefit having consolidated; run a pilot project with that data, to ensure your processes work and you are seeing the business benefits you expect; and then start adding other sources, as time and resources permit. This approach means your project will take longer and possibly cost more, but the risk is lower. This approach also lets you start with a few organizations and add more as the project demonstrates success, instead of trying to get everybody on board from the start.

Another factor to consider when merging your source data into the master list is privacy. When customers become part of the customer master, their information might be visible to any of the applications that have access to the customer master. If the customer data was obtained under a privacy policy that limited its use to a particular application, you might not be able to merge it into the customer master. You might want to add a lawyer to your MDM planning team.

At this point, if your goal was to produce a list of master data, you are done. Print it out or burn it to a CD, and move on. If you want your master data to stay current as data is added and changed, you will have to develop infrastructure and processes to manage the master data over time. The next section provides some options on how to do just that.

#How Do I Maintain a Master List?

There are many different tools and techniques for managing and using master data. We will cover three of the more common scenarios here:

• Single-copy approach - In this approach, there is only one master copy of the master data. All additions and changes are made directly to the master data. All applications that use master data are rewritten to use the new data instead of their current data. This approach guarantees consistency of the master data, but in most cases it's not practical. Modifying all your applications to use a new data source with a different schema and different data is, at least, very expensive; if some of your applications are purchased, it might even be impossible.
• Multiple copies, single maintenance - In this approach, master data is added or changed in the single master copy of the data, but changes are sent out to the source systems in which copies are stored locally. Each application can update the parts of the data that are not part of the master data, but they cannot change or add master data. For example, the inventory system might be able to change quantities and locations of parts, but new parts cannot be added, and the attributes that are included in the product master cannot be changed. This reduces the number of application changes that will be required, but the applications will minimally have to disable functions that add or update master data. Users will have to learn new applications to add or modify master data, and some of the things they normally do will not work anymore.
• Continuous merge - In this approach, applications are allowed to change their copy of the master data. Changes made to the source data are sent to the master, where they are merged into the master list. The changes to the master are then sent to the source systems and applied to the local copies. This approach requires few changes to the source systems; if necessary, the change propagation can be handled in the database, so no application code is changed. On the surface, this seems like the ideal solution. Application changes are minimized, and no retraining is required. Everybody keeps doing what they are doing, but with higher-quality, more complete data. This approach does have several issues:
• Update conflicts are possible and difficult to reconcile. What happens if two of the source systems change a customer's address to different values? There's no way for the MDM system to decide which one to keep, so intervention by the data steward is required; in the meantime, the customer has two different addresses. This must be addressed by creating data-governance rules and standard operating procedures, to ensure that update conflicts are reduced or eliminated.
• Additions must be remerged. When a customer is added, there is a chance that another system has already added the customer. To deal with this situation, all data additions must go through the matching process again to prevent new duplicates in the master.
• Maintaining consistent values is more difficult. If the weight of a product is converted from pounds to kilograms and then back to pounds, rounding can change the original weight. This can be disconcerting to a user who enters a value and then sees it change a few seconds later.

In general, all these things can be planned for and dealt with, making the user's life a little easier, at the expense of a more complicated infrastructure to maintain and more work for the data stewards. This might be an acceptable trade-off, but it's one that should be made consciously.

#Versioning and Auditing

No matter how you manage your master data, it's important to be able to understand how the data got to the current state. For example, if a customer record was consolidated from two different merged records, you might need to know what the original records looked like, in case a data steward determines that the records were merged by mistake and really should be two different customers. The version management should include a simple interface for displaying versions and reverting all or part of a change to a previous version. The normal branching of versions and grouping of changes that source-control systems use can also be very useful for maintaining different derivation changes and reverting groups of changes to a previous branch.

Data stewardship and compliance requirements will often include a way to determine who made each change and when it was made. To support these requirements, an MDM system should include a facility for auditing changes to the master data. In addition to keeping an audit log, the MDM system should include a simple way to find the particular change you are looking for. An MDM system can audit thousands of changes a day, so search and reporting facilities for the audit log are important.

#Hierarchy Management

In addition to the master data itself, the MDM system must maintain data hierarchies—for example, bill of materials for products, sales territory structure, organization structure for customers, and so forth. It's important for the MDM system to capture these hierarchies, but it's also useful for an MDM system to be able to modify the hierarchies independently of the underlying systems. For example, when an employee moves to a different cost center, there might be impacts to the Travel and Expense system, payroll, time reporting, reporting structures, and performance management. If the MDM system manages hierarchies, a change to the hierarchy in a single place can propagate the change to all the underlying systems. There might also be reasons to maintain hierarchies in the MDM system that do not exist in the source systems. For example, revenue and expenses might need to be rolled up into territory or organizational structures that do not exist in any single source system. Planning and forecasting might also require temporary hierarchies to calculate "what if" numbers for proposed organizational changes. Historical hierarchies are also required in many cases to roll up financial information into structures that existed in the past, but not in the current structure. For these reasons, a powerful, flexible hierarchy-management feature is an important part of an MDM system.

Conclusion

The recent emphasis on regulatory compliance, SOA, and mergers and acquisitions has made the creating and maintaining of accurate and complete master data a business imperative. Both large and small businesses must develop data-maintenance and governance processes and procedures, to obtain and maintain accurate master data. While it's easy to think of master-data management as a technological issue, a purely technological solution without corresponding changes to business processes and controls will likely fail to produce satisfactory results. This paper has covered the reasons for adopting master-data management, the process of developing a solution, and several options for the technological implementation of the solution. Future papers in this series will explain the technological and procedural issues that must be resolved to implement an MDM system.

...

Original Source : Microsoft MSDN

The What, Why, and How of Master Data Management - Part 1 - By Microsoft

Introduction

The pain that organizations are experiencing around consistent reporting, regulatory compliance, strong interest in Service-Oriented Architecture (SOA), and Software as a Service (SaaS) has prompted a great deal of interest in Master Data Management (MDM). This paper explains what MDM is, why it is important, and how to manage it, while identifying some of the key MDM management patterns and best practices that are emerging. This paper is a high-level treatment of the problem space. In subsequent papers, we will drill down into the technical and procedural issues involved in Master Data Management.

What Is Master Data?

Most software systems have lists of data that are shared and used by several of the applications that make up the system. For example, a typical ERP system as a minimum will have a Customer Master, an Item Master, and an Account Master. This master data is often one of the key assets of a company. It's not unusual for a company to be acquired primarily for access to its Customer Master data.

Rudimentary Definitions

There are some very well-understood and easily identified master-data items, such as "customer" and "product." In fact, many define master data by simply reciting a commonly agreed upon master-data item list, such as: customer, product, location, employee, and asset. But how you identify elements of data that should be managed by a master-data management system is much more complex and defies such rudimentary definitions. In fact, there is a lot of confusion around what master data is and how it is qualified, necessitating a more comprehensive treatment.

There are essentially five types of data in corporations:

• Unstructured - This is data found in e-mail, white papers like this, magazine articles, corporate intranet portals, product specifications, marketing collateral, and PDF files.
• Transactional - This is data related to sales, deliveries, invoices, trouble tickets, claims, and other monetary and non-monetary interactions.
• Metadata - This is data about other data and may reside in a formal repository or in various other forms such as XML documents, report definitions, column descriptions in a database, log files, connections, and configuration files.
• Hierarchical - Hierarchical data stores the relationships between other data. It may be stored as part of an accounting system or separately as descriptions of real-world relationships, such as company organizational structures or product lines. Hierarchical data is sometimes considered a super MDM domain, because it is critical to understanding and sometimes discovering the relationships between master data.
• Master - Master data are the critical nouns of a business and fall generally into four groupings: people, things, places, and concepts. Further categorizations within those groupings are called subject areas, domain areas, or entity types. For example, within people, there are customer, employee, and salesperson. Within things, there are product, part, store, and asset. Within concepts, there are things like contract, warrantee, and licenses. Finally, within places, there are office locations and geographic divisions. Some of these domain areas may be further divided. Customer may be further segmented, based on incentives and history. A company may have normal customers, as well as premiere and executive customers. Product may be further segmented by sector and industry. The requirements, life cycle, and CRUD cycle for a product in the Consumer Packaged Goods (CPG) sector is likely very different from those of the clothing industry. The granularity of domains is essentially determined by the magnitude of differences between the attributes of the entities within them.

Deciding What to Manage

While identifying master data entities is pretty straightforward, not all data that fits the definition for master data should necessarily be managed as such. This paper narrows the definition of master data to the following criteria, all of which should be considered together when deciding if a given entity should be treated as master data.

Behavior

Master data can be described by the way that it interacts with other data. For example, in transaction systems, master data is almost always involved with transactional data. A customer buys a product. A vendor sells a part, and a partner delivers a crate of materials to a location. An employee is hierarchically related to their manager, who reports up through a manager (another employee). A product may be a part of multiple hierarchies describing their placement within a store. This relationship between master data and transactional data may be fundamentally viewed as a noun/verb relationship. Transactional data capture the verbs, such as sale, delivery, purchase, email, and revocation; master data are the nouns. This is the same relationship data-warehouse facts and dimensions share.

Life Cycle

Master data can be described by the way that it is created, read, updated, deleted, and searched. This life cycle is called the CRUD cycle and is different for different master-data element types and companies. For example, how a customer is created depends largely upon a company's business rules, industry segment, and data systems. One company may have multiple customer-creation vectors, such as through the Internet, directly through account representatives, or through outlet stores. Another company may only allow customers to be created through direct contact over the phone with its call center. Further, how a customer element is created is certainly different from how a vendor element is created. The following table illustrates the differing CRUD cycles for four common master-data subject areas.

Cardinality

As cardinality (the number of elements in a set) decreases, the likelihood of an element being treated as a master-data element—even a commonly accepted subject area, such as customer—decreases. For example, if a company has only three customers, most likely they would not consider those customers master data—at least, not in the context of supporting them with a master-data management solution, simply because there is no benefit to managing those customers with a master-data infrastructure. Yet, a company with thousands of customers would consider Customer an important subject area, because of the concomitant issues and benefits around managing such a large set of entities. The customer value to each of these companies is the same. Both rely upon their customers for business. One needs a customer master-data solution; the other does not. Cardinality does not change the classification of a given entity type; however, the importance of having a solution for managing an entity type increases as the cardinality of the entity type increases.

Lifetime

Master data tends to be less volatile than transactional data. As it becomes more volatile, it typically is considered more transactional. For example, some might consider "contract" a master-data element. Others might consider it a transaction. Depending on the lifespan of a contract, it can go either way. An agency promoting professional athletes might consider their contracts as master data. Each is different from the other and typically has a lifetime of greater than a year. It may be tempting to simply have one master-data item called "athlete." However, athletes tend to have more than one contract at any given time: one with their teams and others with companies for endorsing products. The agency would need to manage all those contracts over time, as elements of the contract are renegotiated or athletes traded. Other contracts—for example, contracts for detailing cars or painting a house—are more like a transaction. They are one-time, short-lived agreements to provide services for payment and are typically fulfilled and destroyed within hours.

Complexity

Simple entities, even valuable entities, are rarely a challenge to manage and are rarely considered master-data elements. The less complex an element, the less likely the need to manage change for that element. Typically, such assets are simply collected and tallied. For example, Fort Knox likely would not track information on each individual gold bar stored there, but rather only keep a count of them. The value of each gold bar is substantial, the cardinality high, and the lifespan long; yet, the complexity is low.

Value

The more valuable the data element is to the company, the more likely it will be considered a master data element. Value and complexity work together.

Volatility

While master data is typically less volatile than transactional data, entities with attributes that do not change at all typically do not require a master-data solution. For example, rare coins would seem to meet many of the criteria for a master-data treatment. A rare-coin collector would likely have many rare coins. So, cardinality is high. They are valuable. They are also complex. For example, rare coins have a history and description. There are attributes, such as condition of obverse, reverse, legend, inscription, rim, and field. There are other attributes, such as designer initials, edge design, layers, and portrait.

Yet, rare coins do not need to be managed as a master-data item, because they don't change over time—or, at least, they don't change enough. There may need to be more information added, as the history of a particular coin is revealed or if certain attributes must be corrected. But, generally speaking, rare coins would not be managed through a master-data management system, because they are not volatile enough to warrant a solution.

Reuse

One of the primary drivers of master-data management is reuse. For example, in a simple world, the CRM system would manage everything about a customer and never need to share any information about the customer with other systems. However, in today's complex environments, customer information needs to be shared across multiple applications. That's where the trouble begins. Because—for a number of reasons—access to a master datum is not always available, people start storing master data in various locations, such as spreadsheets and application private stores. There are still reasons, such as data-quality degradation and decay, to manage master data that is not reused across the enterprise. However, if a master-data entity is reused in multiple systems, it's a sure bet that it should be managed with a master-data management system.

To summarize, while it is simple to enumerate the various master-data entity types, it is sometimes more challenging to decide which data items in a company should be treated as master data. Often, data that does not normally comply with the definition for master data may need to be managed as such, and data that does comply with the definition may not. Ultimately, when deciding on what entity types should be treated as master data, it is better to categorize them in terms of their behavior and attributes within the context of the business needs than to rely on simple lists of entity types.

Why Should I Manage Master Data?

Because it is used by multiple applications, an error in master data can cause errors in all the applications that use it. For example, an incorrect address in the customer master might mean orders, bills, and marketing literature are all sent to the wrong address. Similarly, an incorrect price on an item master can be a marketing disaster, and an incorrect account number in an Account Master can lead to huge fines or even jail time for the CEO—a career-limiting move for the person who made the mistake!

Here is a typical master-data horror story: A credit-card customer moves from 2847 North 9th St. to 1001 11th St. North. The customer changed his billing address immediately, but did not receive a bill for several months. One day, the customer received a threatening phone call from the credit-card billing department, asking why the bill has not been paid. The customer verifies that they have the new address, and the billing department verifies that the address on file is 1001 11th St. N. The customer asks for a copy of the bill, to settle the account. After two more weeks without a bill, the customer calls back and finds the account has been turned over to a collection agency. This time, they find out that even though the address in the file was 1001 11th St. N, the billing address is 101 11th St. N. After a bunch of phone calls and letters between lawyers, the bill finally gets resolved and the credit-card company has lost a customer for life. In this case, the master copy of the data was accurate, but another copy of it was flawed. Master data must be both correct and consistent.

Even if the master data has no errors, few organizations have just one set of master data. Many companies grow through mergers and acquisitions. Each company you acquire comes with its own customer master, item master, and so forth. This would not be bad if you could just Union the new master data with your current master data, but unless the company you acquire is in a completely different business in a faraway country, there's a very good chance that some customers and products will appear in both sets of master data—usually, with different formats and different database keys. If both companies use the Dun & Bradstreet number or Social Security number as the customer identifier, discovering which customer records are for the same customer is a straightforward issue; but that seldom happens. In most cases, customer numbers and part numbers are assigned by the software that creates the master records, so the chances of the same customer or the same product having the same identifier in both databases is pretty remote. Item masters can be even harder to reconcile, if equivalent parts are purchased from different vendors with different vendor numbers.

Merging master lists together can be very difficult. The same customer may have different names, customer numbers, addresses, and phone numbers in different databases. For example, William Smith might appear as Bill Smith, Wm. Smith, and William Smithe. Normal database joins and searches will not be able to resolve these differences. A very sophisticated tool that understands nicknames, alternate spellings, and typing errors will be required. The tool will probably also have to recognize that different name variations can be resolved, if they all live at the same address or have the same phone number. While creating a clean master list can be a daunting challenge, there are many positive benefits to your bottom line from a common master list:

• A single, consolidated bill saves money and improves customer satisfaction.
• Sending the same marketing literature to a customer from multiple customer lists wastes money and irritates the customer.
• Before you turn a customer account over to a collection agency, it would be good to know if they owe other parts of your company money or, more importantly, that they are another division's biggest customer.
• Stocking the same item under different part numbers is not only a waste of money and shelf space, but can potentially lead to artificial shortages.

The recent movements toward SOA and SaaS make Master Data Management a critical issue. For example, if you create a single customer service that communicates through well-defined XML messages, you may think you have defined a single view of your customers. But if the same customer is stored in five databases with three different addresses and four different phone numbers, what will your customer service return? Similarly, if you decide to subscribe to a CRM service provided through SaaS, the service provider will need a list of customers for their database. Which one will you send them?

For all these reasons, maintaining a high-quality, consistent set of master data for your organization is rapidly becoming a necessity. The systems and processes required to maintain this data are known as Master Data Management.

Continue : Part 2 - What is Master Data Management? & Conclusion

Data Migration Overview - By Javlin Data Solutions

Data Migration is the process of transferring data from one system to another while changing the storage, database or application. In reference to the ETL (Extract-Transform-Load) process, data migration always requires at least Extract and Load steps.

Typically data migration occurs during an upgrade of existing hardware or transfer to a completely new system. Examples include: migration to or from hardware platform; upgrading a database or migrating to new software; or company-mergers when the parallel systems in the two companies need to be merged into one. There are three main options to accomplish data migration:

1. Merge the systems from the two companies into a brand new one.
2. Migrate one of the systems to the other one.
3. Leave the systems as they are but create a common view on top of them - a data warehouse.

Data Migration Challenges

Let us describe the data migration challenges in little more detail. Data migration can be a simple process, however there are challenges one may face in implementation.

#Storage Migration
Storage migration can be handled in a manner transparent to the application so long as the application uses only general interfaces to access the data. In most systems this is not an issue. However, careful attention is necessary for old applications running on proprietary systems. In many cases, the source code of the application is not available and the application vendor may not be in market anymore. In such cases storage migration is rather tricky and should be properly tested before releasing the solution into production.

#Database Migration

Database migration is rather straight forward, assuming the database is used just as storage. It "only" requires moving the data from one database to another. However, even this may be a difficult task. The main issues one may encounter include:

1. Unmatched data types (number, date, sub-records)
2. Different character sets (encoding)

Different data types can be handled easily by approximating the closest type from the target database to maintain data integrity. If a source database supports complex data formats (e.g. sub-record), but the target database does not, amending the applications using the database is necessary. Similarly, if the source database supports different encoding in each column for a particular table but the target database does not, the applications using the database need to be thoroughly reviewed.

When a database is used not just as data storage, but also to represent business logic in the form of stored procedures and triggers, close attention must be paid when performing a feasibility study of the migration to target database. Again, if the target database does not support some of the features, changes may need to be implemented by applications or by middleware software.

ETL tools are very well suited for the task of migrating data from one database to another i. Using the ETL tools is highly advisable particularly when moving the data between the data stores which do not have any direct connection or interface implemented.

#Application Migration
If we take a step back to previous two cases, you may notice that the process is rather straight forward. This however, is extremely uncommon in the case of application migration. The reason is that the applications, even when designed by the same vendor, store data in significantly different formats and structures which make simple data transfer impossible. The full ETL process is a must as the Transformation step is not always straight forward. Of course, application migration can and usually does include storage and database migration as well. The advantage of an ETL tool in this instance is its ready-to-use connectivity to disparate data sources/targets.

Difficulty may occur when migrating data from mainframe systems or applications using proprietary data storage. Mainframe systems use record based formats to store data. Record based formats are easy to handle; however, there are often optimizations included in the mainframe data storage format which complicate data migration. Typical optimizations include binary coded decimal number storage, non-standard storing of positive/negative number values, or storing the mutually exclusive sub-records within a record. Let us consider a library data warehouse as an example. There are two types of publications - books and articles. The publication can be either a book or an article but not both. There are different kinds of information stored for books and articles. The information stored for a book and an article are mutually exclusive. Hence, when storing a book, the data used has a different sub-record format for a book and an article while occupying the same space. Still the data is stored using rather standard encoding. On the Contrary, proprietary data storage makes the Extract step even more complicated. In both cases, the most efficient way to extract data from the source system is performing the extraction in the source system itself; then converting the data into a printable format which can be parsed later using standard tools.

#Character Encoding
Most of the systems developed on PC-based platform use ASCII encoding or national extension based on ASCII. The latest one is UTF-8 which keeps ASCII mapping for alpha and numerical characters but enables storage of characters for most of the national alphabets including Chinese, Japanese and Russian. Mainframe systems are mostly based on EBCDIC encoding which is incompatible with ASCII and conversion is required to display the data. ETL tools should support the conversions between character sets, including EBCDIC.

...

Source : DataIntegration.Info (Javlin - Data Solutions)

Master Data Management - By Javlin Data Solutions

Master Data Management (MDM) represents a set of tools and processes used by an enterprise to consistently manage their non-transactional data. MDM is usually applied to entities like Customer, Client, Product, Account and internal reference data. MDM also enables easy maintenance of data lineage and history for audit purposes.

Issues

In the enterprise there are several systems managing the same data; the role of MDM is to centralize the data management to only one master copy of the data item which is then synchronized to all applications using the data. Using this approach, when referring to (for example) a customer within the enterprise, all systems are referring to the same customer.

There are basically two reasons why there are duplicated data which are inconsistent:

• The production systems within an enterprise, when implemented, have not been designed to be a part of larger set of production systems with which they should cooperate. Therefore, each system manages data on its own.
• The branches or departments of the company exist on their own without close cooperation with other departments. For example, the mortgage department deals with customers and manages the mortgage contracts. While the marketing department plans a promotion on mortgages. If the two departments do not cooperate (share the data), the marketing department may offer a mortgage to a customer who already has a mortgage. This is both a waste of money on the promotion as well as annoying to the customer.
• Company acquisitions or mergers are another example when an enterprise gets several parallel systems managing similar and sometimes overlapping data.

Solutions

To handle the issues mentioned above, the common baseline for Master Data Management solutions comprises the following processes:

• Source identification - the 'system of record' needs to be identified first. If the same record is stored in multiple systems, the system which holds the most relevant copy (most valid, actual, or complete) of that record is referred to as a 'system of record'.
• Data collection - the data needs to be collected from various sources as some sources may attach a new piece of information, while dropping pieces which they are not interested in.
• Transformation - the transformation step takes place both during the input, while data are converted into a format for MDM processing, as well as on the output when distributing the master records back to the particular systems and applications.
• Data consolidation - the records from various systems which represent the same physical entity are consolidated into one record - a master record. The record is assigned a version number to enable a mechanism to check which version of record is being used in particular systems.
• Data deduplication - often there are separate records in the company's systems, which in fact identify the same customer. For example, the bank may have a record identifying a customer while the bank's insurance subsidiary or department maintains a separate database of customers having a different record for the same customer. It is vital that these two records are deduplicated and maintained as one master record.
• Error detection - based on the rules and metrics, the incomplete records or records containing inconsistent data should be identified and sent to their respective owners before publishing them to all the other applications. Providing erroneous data may compromise credibility of the company's MDM.
• Data correction - related to error detection, this step notifies the owner of the data record that there is a need to review the record manually.
• Data distribution/synchronization - the master records are distributed to the systems in the enterprise. The goal is that all the systems are using the same version of the record as soon as possible after the publication of the new record.

Process

In the previous paragraphs, we have mentioned that each data record has to be assigned its owner or steward - a person who understands the data and is responsible for maintaining the record. The steward needs to be from the business side of the company. The reason, of course, is that only a business person understands the data and can make decisions about the data consolidation, updates, corrections and validity. On the other hand, the actual processing can be made available to either the business user via GUI or the IT department.

...

Source : DataIntegration.Info (Javlin - Data Solutions)