General Information
| Job Description | DATA SYS ANL 3 | Working Title | Data Engineer |
|---|---|---|---|
| Job Code | 007199 | Grade | 23 |
| Department Name | Data Engineering Dept - D02044 | Department Head | Michael Kennedy |
| Supervisor | Sattanathan Ponnuswamy | Effective Date | 04/16/2024 |
Position(s) Directly Supervised
| Job Code | Title | FTE |
|---|
Generic Scope
| Experienced professional who knows how to apply theory and put it into practice with in-depth understanding of the professional field; independently performs the full range of responsibilities within the function; possesses broad job knowledge; analyzes problems/issues of diverse scope and determines solutions. |
Custom Scope
| Applies skills as a seasoned, experienced data/information management professional with full understanding of industry practices and campus/medical center/OP and department methodologies, policies and procedures to resolve complex and wide ranging issues where analyses of situations or data requires a review of a variety of factors. Demonstrates competency in selecting methods and techniques to obtain solutions. |
Department Custom Scope
| The Data Engineer will be responsible for creating and maintaining data pipelines between Google BigQuery and vended or UCR hosted sources in support of a campus wide data warehouse initiative. The successful candidate will utilize various GCP components and tools (BigQuery, Cloud Functions, Cloud Storage, Pub/Sub, etc.) combined with Python and GitLab to facilitate the population and ongoing maintenance of the data warehouse. Additionally, the Data Engineer would be responsible for assisting in the investigation and planning of data source ingestion along with assisting analysts and business intelligence team members facilitating the utilization of the data warehouse by interested campus partners and other data consumers. |
Education & Experience Requirements
Education Requirements
| Degree | Requirement |
|---|---|
| Bachelor's degree in related area and/or equivalent experience/training. | Required |
Experience Requirements
| Experience | Requirement |
|---|---|
| 4 - 7 years of related experience. | Required |
| Experience with Google Cloud Platform tools (BigQuery, Cloud Functions, Logging, Cloud IAM, Pub/Sub, Cloud Scheduler, Cloud Storage). | Required |
| Minimum of 4 years of working experience with GCP data centric tools. | Preferred |
| Experience with data translation tools (DBT/Google Cloud Dataform) data visualization tools Looker/Looker Studio/Tableau. | Preferred |
| Experience using source control to maintain and deploy changes in Cloud Function and ELT Code. Experience in REST API usage and Python code. | Required |
License Requirements
Certification Requirements
| Certification | Requirement |
|---|---|
| Google Cloud Platform (GCP): Professional Data Engineer | Preferred |
Educational Condition Requirements
| Condition | Requirement |
|---|
Key Responsibilities
| Description | % Time |
|---|---|
| Implements related business processes, develops logical data models, dimensional data models and schemas for the scope of a project or a component of a project. Develops data mapping specifications and extract-transform-load (ETL) designs. | 20 |
| Creates, automates, and maintains data pipelines utilizing available tools. (BigQuery, Cloud Functions, Cloud Storage, Pub/Sub, Python, GitLab) Manage overall pipeline orchestration. Data Warehouse performance optimization. | 20 |
| Applies professional and broadly defined business analysis and data/information administration and data use concepts to manage and perform complex assignments. | 15 |
| Independently analyzes complex information requirements and documents them using formal data definition techniques and naming standards. Gathers, analyzes, prepares and summarizes information from subject matter experts and technical personnel. Researches relevant problem-domain literature and information standards and develops recommendations for data management resource plans including approaches, trends, sources and uses. | 15 |
| Independently designs information transformation processes. Designs and implements capture of metadata from technical processes such as software design and database management. Performs complex data management or resource research and studies for a variety of constituents and plans metadata acquisition for sets of unstructured data objects such as collections of images, documents or recordings. | 10 |
| Writes SQL queries. Leverages REST APIs. Develops code using Python, Node.js, Nest.js and other programming languages. | 10 |
| Investigates datasets that may not be well documented to assist in creating data models and views. | 5 |
| Develops full data/information documentation for highly complex small and medium-size projects. | 5 |
Knowledge, Skills & Abilities
| Knowledge/Skill/Ability | Requirement |
|---|---|
| Strong organizational skills. | Required |
| Ability to prepare data models unassisted. | Required |
| Ability to represent relevant information in abstract models. Critical thinking skills and attention to detail. | Required |
| Strong analytical and design skills, including the ability to abstract information requirements from real-world processes to understand information flows in computer systems. | Required |
| Ability to represent relevant information in abstract models. | Required |
| Thorough knowledge of data management systems, practices and standards. | Required |
| Familiarity with logical data design and data mapping or data conversion. | Required |
| Familiarity with data model patterns in one or more common business or academic domains. | Required |
| Self-motivated and works independently and as part of a team. | Required |
| Demonstrated ability to work with others from diverse backgrounds. Demonstrated effective communication and interpersonal skills. Demonstrated service orientation skills. | Required |
| Demonstrated strong problem-solving skills. Able to learn effectively and meet deadlines. | Required |
| Familiarity with data quality and governance issues and requirements. | Required |
| Demonstrated ability to communicate technical information to technical and non-technical personnel at various levels in the organization. | Required |
| Thorough knowledge of relevant rules and regulations. | Required |
| Excellent planning and problem-solving skills. | Required |
| Proficiency in SQL query language. Expertise in data structures, manipulation, and data design. Ability to investigate datasets that may not be well documented. Proficiency in Python or similar language. | Required |
| Ability to manage multiple projects/tasks with competing priorities and keep leadership informed of changes. | Required |
| Ability to think outside the documentation and apply techniques that are not explicitly referenced. | Preferred |
| Knowledge of OAuth process and flow. | Preferred |
Special Requirements & Conditions
| Special Condition | Requirement |
|---|---|
| Must pass a background check. | Required |
| Occasional travel for university related business meetings, conferences and/or professional development. | Required |
| Travel Outside of Normal Business Hours | Required |
Other Special Requirements & Conditions
|
|
Level of Supervision Received
| General Supervision |
Environment
Working Environment
| Remote (working from home) with occasional visits to campus as necessary. |
Other Requirements
Items Used
|
Physical Requirements
|
Mental Requirements
|
Environmental Requirements
|
Critical Position
| Is Critical Position: Yes |