SQL SERVER – Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) – Sensitivity of Suggestion

Earlier I talked about what kind of questions, I do not like when I get asked. Today we will go over the question which I like when I get asked the same. One of the reader practices various steps in my earlier blog post Step by Step Guide to Beginning Data Quality Services in SQL Server 2012 – Introduction to DQS. While reading the blog post he noticed that Data Quality Services is not providing very helpful suggestions. He wrote an email to me about it. Let us go over his email.

“Pinal,

I noticed in one of your images that DQS is not providing very helpful suggestions. First of all DQS should be able to make intelligent guesses and make the necessary correction by itself. If it cannot do the same, in that case, it should give us intelligent suggestions but in the image included here, I see the suggestions are not there as well.

SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-6

Why is it so? Would you please tell me how to increase the numbers of suggestion? I do understand this may not be preferable solution in many case but all the business cases go on it depends. There are cases when the high sensitivity required and there are cases when higher sensitivities are not required. I would like to seek your help here.

–Sriram MD”

This is indeed a great question. I see that Sriram understands that every system is different and every application has a different need. I will not have to tell him this most important concept. The question is about how to change the sensitivity of suggestions for correction in DQS. Well, this option is available under the configuration tab in the DQS client.

SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-1

Once you click on Configuration you will see the following screen. Click the Tab of General Settings. You will see the section of Interactive Cleansing. Under this second there is the first option of “Min score for suggestions”. As this is set to 0.7 every suggestion which matches 0.7 probabilities or higher probability are displayed under the suggestion tab.

SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-2
You can see in the following image that there is no suggestion as the min score for suggestions is set to 0.7 and there is no record which qualifies to that much confidence.
SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-3

Now let us change the value of Min Score for suggestion to 0.5.

SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-4
The lower value increased the confidence of DQS to give further suggestion to values which are over 0.5. However, in our case the suggestions which it provides are also accurate. This may not be true for your sample. Every sample is different so you should manually review it before approving them.
SQL SERVER - Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) - Sensitivity of Suggestion  dqs-5

I guess, this is a simple blog post to demonstrate how to change the confidence value for the suggestions which Data Quality Services provides. Use this feature with care and always tune it according to your datasets and record diversity.

Reference: Pinal Dave (https://blog.sqlauthority.com)

 

 

 

,
Previous Post
SQL SERVER – Unable to DELETE Project in Data Quality Projects (DQS)
Next Post
SQL SERVER – Why Do We Need Data Quality Services – Importance and Significance of Data Quality Services (DQS)

Related Posts

1 Comment. Leave new

  • Kannan Chandrasekaran
    March 22, 2013 2:22 am

    Yeah, this is a good question. Even for me the same question arises, because I have some knowledge around fuzzy.. where we used to play with similarity & Confidence %s a lot :)

    Good Question & Best Solution. well explained

    Reply

Leave a Reply

Menu