Product A/B Testing Manager
At Shutterfly, we’re all about people — bringing them together, making them feel welcome, and connecting them to experiences. We make our customers’ memories last a lifetime by capturing, preserving, and sharing them through photography and personalized products. Through our family of brands, trend setting products, cutting edge technology, and best in class customer service, we help our customers, and each other, share life’s joy. Lifetouch has been the professional photography choice for schools and families for more than 80 years. Built on the tradition of “Picture Day” Lifetouch captures smiling faces, preschool through high school graduation. We also reach our customers through additional product lines including portrait studios, yearbooks, sports, and events.
Lifetouch is looking for a Product A/B Testing Manager who will own and manage our testing & personalization initiatives across the entire product roadmap. This role will use the power of testing data to optimize products and enhance our consumer experience.
This role is responsible for running the A/B testing program, leading experiments on our digital platforms and reporting on results. The role works in close coordination with a variety of partners, such as product, marketing, UX and engineering teams. Our mission is to provide expertise, guidance and analysis throughout the A/B testing life cycle so that our businesses can make critical improvements to our digital experiences.
The Product A/B Testing Manager must have strong knowledge of testing and optimization best practices, Adobe Target experience, data analysis, and understanding of development processes. This role will focus on managing test idea intake, prioritization of requests, developing Testing Roadmap and updating Product Management tools to ensure awareness and coordinate activities.
What You'll Do Here:
• Drive and develop the experimentation process from test design, through execution and analysis. Develop appropriate tagging strategy for A/B Testing initiatives
• Coordinate with partners to collect, refine and prioritize all A/B test ideas that are added to the roadmap, with clear test description, hypothesis, important metrics, segmentation and more
• Lead and collaborate with teams on ideation, creation, development, implementation, analysis and recommendations for all A|B testing initiatives. Calculate sample size required and estimated duration for A/B tests based on metrics, hypothesis and volume of traffic
• Collaborate with QA to verify that all test variations are working and tracking before a test launches, and check early data post-launch to detect any issues
• Work with Product & Dev teams on writing JIRA stories for test requirements and participating in Daily scrums or ceremonies as needed.,
• Be the subject matter expert for A|B Testing where you will own, manage and drive the A|B Testing roadmap
• Analyze and communicate the effectiveness/ results of each test or personalization effort, while building out a Hypothesis library for future knowledge sharing and test ideas.
• Support the evolution and expansion of the Adobe Target A/B testing tool implementation on all LIfetouch platforms. Provide implementation guidance and administration of Adobe Target.
• Continually look for process and reporting improvements and implement those changes where applicable
• Handle the creation, upkeep, and documentation of critical Testing artifacts: test plans, business cases, user flow designs, experience design updates, results presentations, etc.
The Skills You'll Bring:
• Experience using Adobe Target or other A/B Testing tools
• Experience using Adobe Analytics
• 3+ years in conversion optimization, A/B and multivariate testing, and /or product optimization.
• Experience using Adobe Launch (nice to have)
• Ability to stand up a Testing program without constant supervision
• Understanding of statistical significance techniques such as Chi-squared test, t-test, ANOVA, correlation, etc.
• Experience and understanding of agile web and product development cycles & working with Jira, Confluence, and/or Aha.
• Proven track record for problem solving and Quality Assurance (QA) for A/B tests and Adobe Target implementations
• Ability to use metrics to manage programs and measure success
• Ability to translate, communicate, and distil metrics into a digestible, actionable recommendations
• Bachelor’s degree in related field of study
It's Not Required But It's Nice To Have:
• Proactive and self-motivated with a strong sense of ownership. Identifies what needs to be done and has a bias toward action.
• Entrepreneurial, digitally savvy, creative and experimental in the approach to problem solving
• Self-starter who is energetic, enthusiastic, adaptable and versatile
• Excellent quantitative skills, written and verbal communication, and presentation skills
• Mastery of implementing and managing successful optimization programs
If this aligns with your career goals, skills and experience - we want to work with you!
** This position is eligible for remote work which may be performed anywhere in the United States except for within the state of Colorado.**