General Information
| Job Description | DATA SYS ANL 4 | Working Title | Senior Data Engineer |
|---|---|---|---|
| Job Code | 000599 | Grade | 25 |
| Department Name | Data Base Administration - D02044 | Department Head | Michael Kennedy |
| Supervisor | Sattanathan Ponnuswamy | Effective Date | 08/21/2023 |
Position(s) Directly Supervised
| Job Code | Title | FTE |
|---|
Generic Scope
| Technical leader with a high degree of knowledge in the overall field and recognized expertise in specific areas; problem-solving frequently requires analysis of unique issues/problems without precedent and/or structure. May manage programs that include formulating strategies and administering policies, processes, and resources; functions with a high degree of autonomy. |
Custom Scope
| Having significant experience, applies advanced data/information management concepts and campus/medical center/OP objectives to resolve highly complex issues where analyses of situations, information or data require an in-depth evaluation of variable factors. Selects methods and techniques to obtain desired results. |
Department Custom Scope
| The Senior Data Engineer will be responsible for creating and maintaining data pipelines between Google BigQuery and vended or UCR hosted sources in support of a campus wide data warehouse initiative. The successful candidate will utilize various GCP components and tools (BigQuery, Cloud Functions, Cloud Storage, Pub/Sub, etc.) combined with Python and GitLab to facilitate the population and ongoing maintenance of the data warehouse. Additionally, the Senior Data Engineer would be responsible for assisting in the investigation and planning of data source ingestion along with assisting analysts and business intelligence team members facilitating the utilization of the data warehouse by interested campus partners and other data consumers. This position will also include mentoring other team members on GCP and other data ingestion facets. |
Education & Experience Requirements
Education Requirements
| Degree | Requirement |
|---|---|
| Bachelor's degree in related area and/or equivalent experience/training. | Required |
Experience Requirements
| Experience | Requirement |
|---|---|
| 6 - 10 years of related experience. | Required |
License Requirements
Certification Requirements
| Certification | Requirement |
|---|---|
| Google Cloud Platform (GCP): Professional Data Engineer | Preferred |
Educational Condition Requirements
| Condition | Requirement |
|---|
Key Responsibilities
| Description | % Time |
|---|---|
Coordinates reviews of data models and data definitions. Develops both conceptual and detailed data models for projects. Develops dimensional models for data subject areas. Creates and manages workflows (including ETL, data curation and others).
|
30 |
Plans and conducts data analysis efforts for highly complex small and medium-sized projects that may include managing partnerships between multiple institutions. Establishes or maintains information documentation standards such as data naming standards and value domain descriptions. Schedules and conducts analysis sessions. Documents ownership, custody and access requirements for data subject areas.
|
20 |
Initiates and designs for diverse date and data systems. Initiates, designs and implements detailed interfaces between computer systems. Initiates, designs, executes and sustains metadata for collections of digital objects.
|
20 |
| Mentor staff in the technologies in use with the data warehouse | 15 |
| Investigates datasets that may not be well documented to assist in creating data models and views. | 10 |
| Develops full data/information documentation for highly complex small and medium-size projects. | 5 |
Knowledge, Skills & Abilities
| Knowledge/Skill/Ability | Requirement |
|---|---|
| Proven modeling and information design skills. | Required |
| Self-motivated and works independently and as part of a team. Strong organizational skills. | Required |
| Familiarity with highly complex data quality and governance issues and requirements. | Required |
| Demonstrated advanced problem-solving skills. Able to learn effectively and meet deadlines. | Required |
| Ability to represent relevant information in abstract models. Critical thinking skills and attention to detail. | Required |
| Demonstrated ability to work with others from diverse backgrounds. Demonstrated effective communication and interpersonal skills. Demonstrated service orientation skills. | Required |
| Experience in data integration and the specification of data services. | Required |
| Ability to understand and model complex knowledge-intensive processes such as scholarly and research processes. | Required |
| Advanced analytical and design skills, including the ability to abstract information requirements from real-world processes to understand information flows in computer systems. | Required |
| Ability to prepare complex data models unassisted. | Required |
| Good negotiation and influencing skills. | Required |
| Advanced knowledge of data management systems, practices and standards. | Required |
| Demonstrated ability to communicate technical information to technical and non-technical personnel at various levels in the organization. | Required |
| Thorough knowledge of relevant rules and regulations. | Required |
| Familiarity with highly complex logical data design and data mapping or data conversion. | Required |
| Experience with Google Cloud Platform tools (BigQuery, Cloud Functions, Logging, Cloud IAM, Pub/Sub, Cloud Scheduler, Cloud Storage). | Required |
| Proficiency in SQL query language. Expertise in data structures, manipulation, and data design. Ability to investigate datasets that may not be well documented. Proficiency in Python or similar language. | Required |
| Experience using source control to maintain and deploy changes in Cloud Function and ELT Code, Experience in REST API usage and Python code | Required |
| Experience with data translation tools (DBT/Google Cloud Dataform) data visualization tools Looker/Looker Studio/Tableau. | Preferred |
| 5+ years of working experience with GCP data centric tools. Excellent planning and problem-solving skills. Ability to think outside the documentation and apply techniques that are not explicitly referenced. Knowledge of OAuth process and flow. Ability to manage multiple projects/tasks with competing priorities and keep leadership informed of changes. Google Cloud Certification: Professional Data Engineer. | Preferred |
Special Requirements & Conditions
| Special Condition | Requirement |
|---|---|
| Must pass a background check. | Required |
| Occasional travel for university related business meetings, conferences and/or professional development. | Required |
| Travel Outside of Normal Business Hours | Required |
Other Special Requirements & Conditions
|
|
Level of Supervision Received
| Direction |
Environment
Working Environment
| Remote (working from home) with occasional visits to campus as necessary. |
Other Requirements
Items Used
|
Physical Requirements
|
Mental Requirements
|
Environmental Requirements
|
Critical Position
| Is Critical Position: Yes |