/
3. DQ Software: Requirements, Procurement and Implementation

Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au

3. DQ Software: Requirements, Procurement and Implementation

Timeline

Dec2022Jan2023FebMarAprMayJunJulAugSepOctNovProcurement CompleteAnalysis CompleteComplete
Analysis
Decision
Implementation

Workshops to Define Must Have Functionality

Gather Provider Capability

Decision

Implementation and Training

 

Key Business Outcomes:

The key business outcome is to ensure any relevant comms (mail, email, SMS) reaches the target audience (e.g. Alumni, Donor, Friend, etc.) thereby increased engagement and uplift in philanthropy donations.

Procurement Policies

No

Description

Document

No

Description

Document

1

Procurement Policy and rules for engagement

Key Requirements

Date

Document

Distribution

Date

Document

Distribution

Dec 5, 2022

Experian Minutes Dec 5, 2022 : https://us02web.zoom.us/rec/share/UCR_5Z_gJDTFfmj_F9Mkt5Hd0ILE7X5-mN4jBNrvaQ4KKm6c0sNRajdrhjtxZxXn.16TeiYaPHTPUx40e?iet=751bmC-FkPd5_2HjN73gza550v1lAAm7jeOFqEbvz_E.AG.TedxnoHtTENAyntBHPTFSRThohKBftuHkG2CECeL8Wav12fjjej4ADcfU287hheIw8x00FjesM573ntMussMPcHkHRE9neTfV828_vX9v-gLLwwh8o-f9db3jkl95294NqXa6g.iIoPTNuTpirXe9qTFf6ITg.f47FW41j0OUjERJt

Passcode: &m.TT0?K

@john.humphreys
@Liz Enright
@Brad Fernandes (Unlicensed)
@AlainGasquet
@Manpreet Sidhu
@Jules Levin

 

 

 

 

Vendor Capability Analysis

 

Ataccama

Informatica

Best in class: https://www.ataccama.com/analyst-research/gartner-magic-quadrant-data-quality-solutions-2022

Gartner ‘challengers’ percentile

Gartner ‘leaders’ percentile

Gartner ‘leaders’ percentile

Dashboards w Historical Progress Graphs

Address Validation

National Change of Address (NCOA)

Mobile Validation

Phone Validation

Email Validation

Consumer Segmentation

Field Level Analysis (isnull, info(), etc

Data Quality Rules

Ease of use out of 10

5 *

8 *

10 *

Access Control Mechanism

Vendor and Cyber Security Assessment

Aperture DSarchived

Cost

$55,000 for 10x licenses

TBA

Current provider - will be using existing licences

Comments

-

 

 

 

Pro’s & Cons moving to Informatica

Pro’s

Cons

Pro’s

Cons

  • I think the tool is quite flexible and UI friendly:-

  • Data quality rule/data dictionary is managed separately and at the same time can be referenced for the data profiling.

  • Data Profiling pipeline looked quite intuitive. (Drag and Drop feature).

  • Data masking is easy (just simply drag that function onto the sheet and choose the field you want to mask).

  • Automatic tagging on target fields, and ability pick source fields and map onto target fields. (again quite viaual and intuitive).

  • Data enrichment capability, will definitely help us to fix lot of missing data.

  • I think Uni people are already using it (Aravind), so will be able to get timely support to resolve any query (if needed).

  • the tool is more complex, not straight forward to use comparing to Aperture DS. with Aperture DS the people who don't have training still could used it after have sometime to look at the exist work flow. example James pick up quick

  • We have to define verify rule while Aperture DS has it build in rule.

  • with aperture ds we could upload the data directly from our computer, but this tool we need to load the file to AWS of some kind before we could access. The connection to Source is not easy.

  • I agree with Manpreet that there are more features like data masking, ....

  • It will be time consuming: as the output of which I saw wasn't usable yet. I think we have to spend lots of times to define and set up to be able to output a passing data file and a failed data file.

  • Informatica also has in built rules, and we can define ours as well. It can automatically pick up data elements and create data dictionary based on the majority of data.

 

 

TAP BI Team User Details

SSO UAT Link:  https://dm-ap.informaticacloud.com/ma/sso/eMfunfYtOoUhLleAo4to6F

Name

Email

Unikey

Albie Roets

albie.roets@sydney.edu.au

aroe8216

Manpreet Sidhu

manpreet.sidhu@sydney.edu.au

msid4459

Yeng Sembrano

yeng.sembrano@sydney.edu.au

msem3353

Polina Nikulina

polina.nikulina@syndey.edu.au

pnik2390 

The-Ho Trang

theho.trang@sydney.edu.au

ttra9492

Binila Russel

binila.russel@sydney.edu.au

brus4814

Pat Dennis

pat.dennis@sydney.edu.au

pden4556

Beha Parsa

Beha.Parsa@sydney.edu.au

Bpar2671

James Allen

James.d.allen@sydney.edu.au

Jall6216

Aneev Singh

Aneev.singh@sydney.edu.au

asin2558

Miranda Adams

miranda.adams@sydney.edu.au

 

 

POC Outcomes:

  1. Ensure that as a minimum it can replicate the Aperture address check functionality → Run a salesforce file through Aperture and Informatica and compare.

  2. Ensure that as a minimum it can replicate the Aperture telephone formatting functionality → Run a salesforce file through Aperture and Informatica and compare.

  3. Ensure that as a minimum it can replicate the Aperture email checking functionality → Run a salesforce file through Aperture and Informatica and compare.

(Desensitised - No PII info in the file)

POC Area

Outcome

Comments

POC Area

Outcome

Comments

Address quality - comparison

 

 

Email Formatting - comparison

 

 

Telephone Formatting - comparison

 

 

 

TAP BI Team User Details

SSO UAT Link:  https://dm-ap.informaticacloud.com/ma/sso/eMfunfYtOoUhLleAo4to6F

Name

Email

Unikey

Albie Roets

albie.roets@sydney.edu.au

aroe8216

Manpreet Sidhu

manpreet.sidhu@sydney.edu.au

msid4459

Yeng Sembrano

yeng.sembrano@sydney.edu.au

msem3353

Polina Nikulina

polina.nikulina@syndey.edu.au

pnik2390 

The-Ho Trang

theho.trang@sydney.edu.au

ttra9492

Binila Russel

binila.russel@sydney.edu.au

brus4814

Pat Dennis

pat.dennis@sydney.edu.au

pden4556

Beha Parsa

Beha.Parsa@sydney.edu.au

Bpar2671

James Allen

James.d.allen@sydney.edu.au

Jall6216

Aneev Singh

Aneev.singh@sydney.edu.au

asin2558

Miranda Adams

miranda.adams@sydney.edu.au

 

 

Meeting Minutes / Actions:

Date

Description

Comments

Date

Description

Comments

 Feb 14, 2023

 Cyber Security to review Workbench https://sydneyuni.atlassian.net/browse/TAP-1216

Get Cyber Security to security review Workbench, @albie.roets , Feb 28, 2023

 Feb 15, 2023

KEY POINTS FROM ATACCAMA DEMO SESSION :-

1. Data Cataloging/Metadata Management to give enterprise-wide view of the data.

2. Ability to integrate the cleansed data into other cloud platforms eg: Snowflake, Azure etc

3. Ability to integrate with JIRA / Confluence

4. Addresses are verified using Google API / Aus Post services.

5. Ability to identify patterns in the data.

6. Supports in-built data quality rules as well as custom rules (tag/column level) based on the data asset.

7. Supports rule based access control.

8. Provides the feature of Data Quality Firewall which helps to prevent entry of poor quality data.

9. Data quality can be tracked down using score card & other metrics.

 

 @Polina Nikulina @AlainGasquet @albie.roets @Brad Fernandes (Unlicensed) FYI

 Mar 3, 2023

Demo of Informatica: ICT → Advancement Services

https://uni-sydney.zoom.us/rec/share/Yx2yMNHeKgnrdRoAargPZ2yv2oKbQ6KvdqYZMM1aXpZ0ch1K1tfOw95yKhK8X7KX.w8OeJ6WLu_JHf7NW
Passcode: +9je@Cc5

Actions:

Provide a list of users with UniKey for access to Infomatica and send the same to Arvind, @albie.roets , Mar 6, 2023
Check if existing Salesforce connection is available to the UAT environment., @Aravind Bobba , Mar 8, 2023
Do a POC in UAT environment once access is granted, @albie.roets , Mar 24, 2023
Support the Advancement team as needed with any implementation steps/ user guides, @Aravind Bobba , Mar 24, 2023

(FYI @AlainGasquet , @Polina Nikulina , @Manpreet Sidhu , @Brad Fernandes (Unlicensed) , @Yeng Sembrano , @Eric Park , @paras.nath , @mahesh.donthireddy , @Jerry Hartard , @Michael Tran (ICT) , @Con Bautista , @md.adil , @Kevin Koay)

 

 

 

 

 

 

 

 

 

 

 

 

 

Related content

2. Data Quality Rules and DQ Dashboards
2. Data Quality Rules and DQ Dashboards
Read with this
2023-12-01 AQ Consultation Session 10
2023-12-01 AQ Consultation Session 10
More like this
Jarvis Data Dictionary / Business Rules
Jarvis Data Dictionary / Business Rules
Read with this
1. Business Rules Meetings Minutes
1. Business Rules Meetings Minutes
Read with this
5.1 SITS-Jarvis Integration BAU Production Defects
5.1 SITS-Jarvis Integration BAU Production Defects
Read with this