Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au
Skip to end of banner
Go to start of banner

DQ Software: Requirements, Procurement and Implementation

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 49 Current »

Timeline

Dec2022Jan2023FebMarAprMayJunJulAugSepOctNovProcurement CompleteAnalysis CompleteComplete
Analysis
Decision
Implementation

Workshops to Define Must Have Functionality

Gather Provider Capability

Decision

Implementation and Training

Key Business Outcomes:

The key business outcome is to ensure any relevant comms (mail, email, SMS) reaches the target audience (e.g. Alumni, Donor, Friend, etc.) thereby increased engagement and uplift in philanthropy donations.

Procurement Policies

No

Description

Document

1

Procurement Policy and rules for engagement

Key Requirements

Date

Document

Distribution

Experian Minutes : https://us02web.zoom.us/rec/share/UCR_5Z_gJDTFfmj_F9Mkt5Hd0ILE7X5-mN4jBNrvaQ4KKm6c0sNRajdrhjtxZxXn.16TeiYaPHTPUx40e?iet=751bmC-FkPd5_2HjN73gza550v1lAAm7jeOFqEbvz_E.AG.TedxnoHtTENAyntBHPTFSRThohKBftuHkG2CECeL8Wav12fjjej4ADcfU287hheIw8x00FjesM573ntMussMPcHkHRE9neTfV828_vX9v-gLLwwh8o-f9db3jkl95294NqXa6g.iIoPTNuTpirXe9qTFf6ITg.f47FW41j0OUjERJt

Passcode: &m.TT0?K

Vendor Capability Analysis

Experian

https://www.experian.com.au/insights-au/data-quality/give-your-data-quality-strategy-a-boost-with-your-own-tailored-data-quality-report/?SP_MID=37954-g&SP_RID=23911092-g&cmpid=APAC-ANZ-DQT-Experian%20Connect%20Newsletter%20Nov%2022_37954&elqTrackId=C325462474FE41A06E21DA0FD2127E61&elq=d9daf931b3af4d81b0f82913f4fde0c5&elqaid=37954&elqat=1&elqCampaignId=18065

Ataccama

Informatica

Best in class: https://www.ataccama.com/analyst-research/gartner-magic-quadrant-data-quality-solutions-2022

Gartner ‘challengers’ percentile

Gartner ‘leaders’ percentile

Gartner ‘leaders’ percentile

Dashboards w Historical Progress Graphs

Do provide, did not configure

Provide Historical Progress Graphs

Provide Historical Progress Graphs

Address Validation

  • No International addresses - extra cost

  • Uses Google API so Unit numbers not good.

  • No International addresses - extra cost

  • Uses Google API so Unit numbers not good.

  • No International addresses - extra cost

  • Uses Google API so Unit numbers not good.

National Change of Address (NCOA)

Australia Post integration - BUT @ extra cost

Australia Post Services included in the licensing fees

Australia Post Services included in the licensing fees

Mobile Validation

Format phone numbers

Format phone numbers

Format phone numbers

Phone Validation

Format phone numbers + validate against HLR (Home Location Register)

Format phone numbers + validate against HLR (Home Location Register)

Format phone numbers + validate against HLR (Home Location Register)

Email Validation

Various Checks

Various Checks

Various Checks

Consumer Segmentation

Mosaic customer profiling

Mosaic customer profiling

Mosaic customer profiling

Field Level Analysis (isnull, info(), etc

Yes

Yes

Yes

Data Quality Rules

Set of Standard Data Quality Rules on Address, Phone and Email

Set of Standard Data Quality Rules on Address, Phone and Email + Customisable for additional rules

Set of Standard Data Quality Rules on Address, Phone and Email + Customisable for additional rules

Ease of use out of 10

5 *

8 *

10 *

Access Control Mechanism

SSO

SSO + Rule based access control

SSO + Rule based access control

Vendor and Cyber Security Assessment

Done

Aperture DS

New Provider, Needs to be done

Existing USyd supplier - Vendor and Cyber Security Assessment already done

Cost

$55,000 for 10x licenses

TBA

Current provider - will be using existing licences

Comments

-

Pro’s & Cons moving to Informatica

Pro’s

Cons

  • I think the tool is quite flexible and UI friendly:-

  • Data quality rule/data dictionary is managed separately and at the same time can be referenced for the data profiling.

  • Data Profiling pipeline looked quite intuitive. (Drag and Drop feature).

  • Data masking is easy (just simply drag that function onto the sheet and choose the field you want to mask).

  • Automatic tagging on target fields, and ability pick source fields and map onto target fields. (again quite viaual and intuitive).

  • Data enrichment capability, will definitely help us to fix lot of missing data.

  • I think Uni people are already using it (Aravind), so will be able to get timely support to resolve any query (if needed).

  • the tool is more complex, not straight forward to use comparing to Aperture DS. with Aperture DS the people who don't have training still could used it after have sometime to look at the exist work flow. example James pick up quick

  • We have to define verify rule while Aperture DS has it build in rule.

  • with aperture ds we could upload the data directly from our computer, but this tool we need to load the file to AWS of some kind before we could access. The connection to Source is not easy.

  • I agree with Manpreet that there are more features like data masking, ....

  • It will be time consuming: as the output of which I saw wasn't usable yet. I think we have to spend lots of times to define and set up to be able to output a passing data file and a failed data file.

  • Informatica also has in built rules, and we can define ours as well. It can automatically pick up data elements and create data dictionary based on the majority of data.

TAP BI Team User Details

SSO UAT Link:  https://dm-ap.informaticacloud.com/ma/sso/eMfunfYtOoUhLleAo4to6F

Name

Email

Unikey

Albie Roets

albie.roets@sydney.edu.au

aroe8216

Manpreet Sidhu

manpreet.sidhu@sydney.edu.au

msid4459

Yeng Sembrano

yeng.sembrano@sydney.edu.au

msem3353

Polina Nikulina

polina.nikulina@syndey.edu.au

pnik2390 

The-Ho Trang

theho.trang@sydney.edu.au

ttra9492

Binila Russel

binila.russel@sydney.edu.au

brus4814

Pat Dennis

pat.dennis@sydney.edu.au

pden4556

Beha Parsa

Beha.Parsa@sydney.edu.au

Bpar2671

James Allen

James.d.allen@sydney.edu.au

Jall6216

Aneev Singh

Aneev.singh@sydney.edu.au

asin2558

Miranda Adams

miranda.adams@sydney.edu.au

 

 

POC Outcomes:

  1. Ensure that as a minimum it can replicate the Aperture address check functionality → Run a salesforce file through Aperture and Informatica and compare.

  2. Ensure that as a minimum it can replicate the Aperture telephone formatting functionality → Run a salesforce file through Aperture and Informatica and compare.

  3. Ensure that as a minimum it can replicate the Aperture email checking functionality → Run a salesforce file through Aperture and Informatica and compare.

(Desensitised - No PII info in the file)

POC Area

Outcome

Comments

Address quality - comparison

Email Formatting - comparison

Telephone Formatting - comparison

TAP BI Team User Details

SSO UAT Link:  https://dm-ap.informaticacloud.com/ma/sso/eMfunfYtOoUhLleAo4to6F

Name

Email

Unikey

Albie Roets

albie.roets@sydney.edu.au

aroe8216

Manpreet Sidhu

manpreet.sidhu@sydney.edu.au

msid4459

Yeng Sembrano

yeng.sembrano@sydney.edu.au

msem3353

Polina Nikulina

polina.nikulina@syndey.edu.au

pnik2390 

The-Ho Trang

theho.trang@sydney.edu.au

ttra9492

Binila Russel

binila.russel@sydney.edu.au

brus4814

Pat Dennis

pat.dennis@sydney.edu.au

pden4556

Beha Parsa

Beha.Parsa@sydney.edu.au

Bpar2671

James Allen

James.d.allen@sydney.edu.au

Jall6216

Aneev Singh

Aneev.singh@sydney.edu.au

asin2558

Miranda Adams

miranda.adams@sydney.edu.au

 

 

Meeting Minutes / Actions:

Date

Description

Comments

 

 Cyber Security to review Workbench TAP-1216 - Getting issue details... STATUS

  • Get Cyber Security to security review Workbench, albie.roets ,

 

KEY POINTS FROM ATACCAMA DEMO SESSION :-

1. Data Cataloging/Metadata Management to give enterprise-wide view of the data.

2. Ability to integrate the cleansed data into other cloud platforms eg: Snowflake, Azure etc

3. Ability to integrate with JIRA / Confluence

4. Addresses are verified using Google API / Aus Post services.

5. Ability to identify patterns in the data.

6. Supports in-built data quality rules as well as custom rules (tag/column level) based on the data asset.

7. Supports rule based access control.

8. Provides the feature of Data Quality Firewall which helps to prevent entry of poor quality data.

9. Data quality can be tracked down using score card & other metrics.

 Polina Nikulina AlainGasquet albie.roets Brad Fernandes (Unlicensed) FYI

 

Demo of Informatica: ICT → Advancement Services

https://uni-sydney.zoom.us/rec/share/Yx2yMNHeKgnrdRoAargPZ2yv2oKbQ6KvdqYZMM1aXpZ0ch1K1tfOw95yKhK8X7KX.w8OeJ6WLu_JHf7NW
Passcode: +9je@Cc5

Actions:

  • Provide a list of users with UniKey for access to Infomatica and send the same to Arvind, albie.roets ,
  • Check if existing Salesforce connection is available to the UAT environment., Aravind Bobba ,
  • Do a POC in UAT environment once access is granted, albie.roets ,
  • Support the Advancement team as needed with any implementation steps/ user guides, Aravind Bobba ,

(FYI AlainGasquet , Polina Nikulina , Manpreet Sidhu , Brad Fernandes (Unlicensed) , Yeng Sembrano , Eric Park , paras.nath , mahesh.donthireddy , Jerry Hartard , Michael Tran (ICT) , Con Bautista , md.adil , Kevin Koay)

 

 

 

 

 

 

 

 

 

 

  • No labels