Manager, Product & Content Experimentation

Job Description

Responsibilities
Introducing Peacock, NBCUniversal's new streaming service that combines timeless shows and movies, exclusive originals, kids programming and current hits, with timely news, sports and pop culture. All together. All in one app.

We are building a world-class team of smart, hungry and fearless professionals who are energized by the possibility of working at the epicenter of content, technology and culture. Join us if you would like to be a part of this exciting initiative.

We are seeking a subject matter expert in experimentation to help lead our A/B/N testing efforts across Product, Commerce, Editorial, and Marketing. With the vast amount of content available from NBCUniversal and the ever-evolving and fast-paced product development of Peacock, we will need a smart and capable individual to manage the A/B and multivariate testing lifecycle, spanning from sifting through large amounts of data for insights into experimentation, generating informed hypotheses and objectives for data-informed prioritization, designing and conducting tests for statistical significance in continuous and iterative optimization, to analyzing testing results and presenting data-driven actions to a multifunctional team of stakeholders in business and technology.

The candidate should have technical experience and statistical skills with randomized controlled experimental design (e.g., experiment goals, testing variables, group randomization, sample size, statistical significance), A/B/N testing toolsets (e.g., Optimizely, Apptimize, Adobe Target, Split), and a programming language for data analysis (e.g., Python, R). This role will sit within the larger Decision Sciences team and partner with other business units including but not limited to: Data Science, Data Engineering, Marketing Analytics, Product, Commerce, Editorial, Marketing, and Ad Solutions.

The goal of the Decision Sciences team is to deliver and champion intelligent use of data that drives a best-in-class customer experience and fuels product growth. The Manager, Product and Content Experimentation, should be able to thrive in a collaborative working environment, gaining buy-in and winning support from various stakeholders while establishing the subject matter expert status as recognized throughout the organization.

Responsibilities include, but are not limited to:
• Managing the randomized controlled experimentation lifecycle spanning from collating A/B/N testing requirements from various business teams, translating business requirements into testing specifications, prioritizing testing backlog based on strategic objectives and tactical goals, collaborating with technical partners to configure and validate tests, launching and monitoring and terminating tests in production, analyzing testing results for statistical significance, to presenting recommended actions to business and technical stakeholders
• Acting as the go-to lead for Product, Commerce, Editorial, and Marketing teams, as well as Engineering group, to answer questions that drive key business decisions through experimental design and execution
• Conducting large numbers of rapid A/B/N experiments in concurrent and sequential tests to drive iterative optimization strategies and tactics in user acquisition, conversion, engagement, and retention, with the goal of continuous CLV (customer lifetime value) improvement
• Ensuring statistical significance of the A/B/N tests by designing, conducting, and analyzing tests administered to exposed groups against control groups
• Performing segmentation analyses on user attributes, content personalization, marketing attribution to inform further testing opportunities
• Evangelizing experimentation efforts by becoming a key voice in technical, statistical, and business terms to enable continuous optimization in Product, Commerce, Editorial, and Marketing

Basic Qualifications:
• BS in statistics, mathematics, computer science, or related field, or equivalent technical work experience
• At least 5 years of experience in an A/B and multivariate testing role, preferably in digital media or video streaming
• Statistical skills in applying hypothesis testing, calculating critical value, defining p-value, and using z-test, t-test, ANOVA, or chi-square test
• Proficiency of at least one programming language for data analysis (i.e., Python, R) and cloud-based relational SQL database management systems (e.g., Google Cloud Platform BigQuery)
• Hands-on experience setting up tests with A/B/N testing tools (e.g., Optimizely, Apptimize, Adobe Target, Split), or demonstrated mastery of experimental design and practical application of same techniques
• Working knowledge of digital analytics platforms (e.g., Adobe Analytics, Google Analytics), data management platforms (e.g., Adobe Audience Manager), campaign management systems (Adobe Campaign Manager), CRM (e.g., Salesforce), and visualization tools (e.g., Tableau, Domo)
• Experience querying data in data warehouse systems (e.g., AWS, Snowflake, BigQuery)
• Ability to translate statistical concepts and probability theories to a general audience

Desired Characteristics:
• An effective team player able to build relationships with a diverse group of business and technical stakeholders
• A confident self-starter able to independently take initiatives and manage competing testing priorities by tight deadlines
• An efficient communicator able to translate business objectives into actionable analyses and analytic results into product recommendations
• A passionate enthusiast of streaming media evolution that is defining the future of entertainment
• A resilient business partner with strong interpersonal skills in motivation, collaboration, and influencing

Notices:
NBCUniversal's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law. NBCUniversal will consider for employment qualified applicants with criminal histories in a manner consistent with relevant legal requirements, including the City of Los Angeles Fair Chance Initiative For Hiring Ordinance, where applicable.

Jobcode: Reference SBJ-g6v6nd-3-143-168-172-42 in your application.