Senior Analyst, Product Experimentation

Job Description

NBCUniversal is one of the world's leading media and entertainment companies.
Our impact is rooted in improving the communities where our employees, customers, and audiences live and work. We have a rich tradition of giving back and ensuring our employees have the opportunity to serve their communities. We champion an inclusive culture and strive to attract and develop a talented workforce to create and deliver a wide range of content reflecting our world.

Job Description
We are seeking a Senior Analyst of experimentation to help support our A/B/N testing efforts across multiple Product teams (Viewing, Commerce, Personalization, and Editorial). With the vast amount of content available from NBCUniversal and the ever-evolving and fast-paced product development of Peacock, we will need a smart and capable individual to support the A/B and multivariate testing lifecycle, spanning from sifting through large amounts of data for insights into experimentation, generating informed hypotheses and objectives for data-informed prioritization, designing and conducting tests for statistical significance in continuous and iterative optimization, to analyzing testing results and presenting data-driven actions to a multifunctional team of stakeholders in business and technology.
The candidate should have technical experience and statistical skills with statistical modeling (e.g., hypothesis testing, multivariate statistics) and randomized controlled experimental design (e.g., KPIs, bucketing, sample size, power, statistical significance), A/B/N testing toolsets (e.g., Optimizely, Apptimize, Adobe Target, Split), a programming language for data analysis (e.g., SQL, Python, R), and full-stack streaming devices (e.g., Xfinity, tvOS, iOS, Android mobile/TV, Roku, Play Station, Xbox, Amazon Fire TV).
This role will sit within the larger Decision Sciences team. The goal of the Decision Sciences team is to deliver and champion intelligent use of data that drives a best-in-class customer experience and fuels product growth. The Senior Analyst, Product Experimentation, therefore, should be able to thrive in a collaborative working environment by partnering with other Decision Sciences teams, including Data Science, Data Engineering, Data Measurement, and other Peacock business units across products, content and marketing.
Responsibilities include, but are not limited to:
• Driving A/B/N experiments in order to directly impact the company's optimization strategy for user acquisition, conversion, engagement, retention, and continuous CLV (customer lifetime value) growth
• Partnering with Product and Marketing teams, as well as Engineering stakeholders, in order to translate business requirements into experiment designs of robust statistical and technical soundness, and ultimately actionable insights into continuous optimization
• Designing, conducting, and analyzing A/B/N tests administered to treatment groups against control and holdout groups via statistical modeling in order to ensure analytical power and significance
• Performing segmentation analyses and statistical modeling on user attributes, content personalization, marketing attribution to inform further testing opportunities
• Developing reporting dashboards and visualizations in order to streamline the experimentation process and to present results to key stakeholders in an elegant fashion
Qualifications
• BS in statistics, mathematics, analytics, or related field, or equivalent technical work experience
• At least two years of experience in an A/B and multivariate testing role, preferably in digital media or video streaming
• Statistical skills in applying hypothesis testing, calculating critical value, defining p-value, and using z-test, t-test, ANOVA, or chi-square test
• Proficiency of at least one programming language for data analysis (i.e., Python, R) and cloud-based relational SQL database management systems (e.g., Google Cloud Platform BigQuery)
• Hands-on experience setting up tests with A/B/N testing tools (e.g., Optimizely, Apptimize, Adobe Target, Split), or demonstrated mastery of experimental design and practical application of same techniques
• Working knowledge of digital analytics platforms (e.g., Adobe Analytics, Google Analytics), data management platforms (e.g., mParticle), and visualization tools (e.g., Tableau, Domo)
• Experience querying data in data warehouse systems (e.g., AWS, Snowflake, BigQuery)
• Ability to translate statistical concepts and probability theories to a general audience
• A basic understanding of complex machine learning frameworks (e.g., neural network, gradient boosting, etc.) is preferred, but not required
Additional Requirements:
Hybrid: This position has been designated as hybrid, generally contributing from the office a minimum of three days per week.
This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks. Learn more about the benefits offered by NBCUniversal by visiting the Benefits page of the Careers website.
Salary range: $85,000 - $115,000

Jobcode: Reference SBJ-4k9ekq-18-97-14-90-42 in your application.