Appen

Beyond the leaderboard. See how real AI performance is measured High quality data for frontier AI Appen has been providing high-quality datasets that power the world's leading AI models for decades. Our end-to-end platform, flexible services and deep expertise ensure the delivery of the high-quality, diverse data that is crucial for building foundation models and enterprise-ready AI applications. Learn moreExplore our platform Powering AI innovation for more than 25 years Appen supports the AI lifecycle Appen provides enterprises with software to collect, curate, fine-tune and monitor traditionally human-driven tasks, creating massive efficiencies through a trustworthy, traceable process. Why choose Appen? Expertise With over 25 years of experience in data and AI, we bring unparalleled expertise to every project. Scale Our services enable us to prepare data at scale, meeting the demands of even the most ambitious AI projects. Quality We ensure that our customers receive high quality data. We understand their tasks and deliver to their requirements. Flexibility We meet our customers where they are. Whether they need our service offerings or leverage the power of our platform to unlock their enterprise data, we can provide a solution for their needs. Innovation As in AI data, we continuously invest in research and technology to push the boundaries of what's possible. Appen’s experience with data across the AI lifecycle will be a critical step forward for helping enterprises accelerate building, deploying and adoption of customizable AI, bringing AI to each and every industry. Learn more Hemant Dhulla Global Head of NVIDIA AI Software In our mission to create world-class artificial intelligence chatbots at record speed, high-quality data sets are essential. Appen is a very important partner in this process because we can rely on them for exactly that: speedy and high-quality datasets that we use to train our AI engine and provide conversational experience. Their global coverage allows us to provide a premium service for our clients, all over the world, in any language. Learn more Ante Stjepanovic Product Marketing Manager at Infobip Appen’s platform is really easy to use. What makes it great is you can reach so many different channels because of its global outreach. Learn more Kenneth Benoit Director of the Data Science Institute, London School of Economics Appen is so fast. Using their platform, we could do overnight what used to take us a month. Appen is wonderfully efficient. Learn more Rick Britt Vice President of AI, CallMiner High Quality Data at Scale Discover how Appen accelerates the development of your AI applications. Talk to an expert Contact us Thank you for getting in touch! We appreciate you contacting Appen. One of our colleagues will get back in touch with you soon! Have a great day! --- Beyond the leaderboard. See how real AI performance is measured We're committed to AI excellence Appen provides accurate and reliable human annotated datasets that fuel AI and machine learning for some of the world’s biggest brands.  With more than 25 years of industry knowledge, Appen powers many of the AI interactions we experience every day. Our crowd solutions and expertise empower businesses to achieve their AI goals and make a significant impact in their industry. Join us as we shape the future of AI and unlock its limitless possibilities. Our leadership team Ryan Kolln Chief Executive Officer Ryan Kolln Chief Executive Officer Ryan Kolln is the Chief Executive Officer and Managing Director of Appen. Ryan brings over 20 years of global experience in technology and telecommunications, along with a deep understanding of Appen’s business and the AI industry. His professional career began as an engineer, with a focus on mobile network data engineering in Australia, Asia and North America. On completion of an MBA from New York University, Ryan joined The Boston Consulting Group (BCG) in 2011 as a strategy consultant. During his time at BCG he specialised in technology and telecommunications and gained deep strategy expertise across a variety of growth and operational topics. Joining Appen AI in 2018 as VP of Corporate Development, he led strategic acquisitions like Figure Eight and Quadrant, and supported the establishment of the China and Federal divisions. Prior to his appointment as CEO, he served as Chief Operating Officer, overseeing global operations and strategy. Helen Attia Chief People Officer Helen Attia Chief People Officer Helen is the Chief People Officer at Appen and is focused on creating a people strategy that supports Appen’s strategic objectives and creating an environment where our people can be their best.  Helen brings over 20 years of Human Resources experience from global organisations primarily in the technology and software sector.  With a passion for Talent Management and Organisational Development, Helen enjoys supporting organisations through periods of growth and change.  Helen holds a Master of Commerce from the University of NSW and an Applied Science degree from the University of Sydney. Justin Miles Chief Financial Officer Justin Miles Chief Financial Officer Justin is responsible for the finance and corporate functions of the company. He brings over 20 years of experience, including extensive exposure to listed companies within the technology and services sectors. He has a passion for supporting fast paced organisations through periods of growth and change. Joining Appen in 2016 he has a deep understanding of Appen’s business. With over 5 years as Vice President Finance, he established the finance structure, systems, and processes that scaled, supported exceptional growth, and successfully executed several finance integrations. Justin was promoted to Interim Chief Financial Officer in August 2023 and was officially appointed Chief Financial Officer in February 2024. Prior to joining Appen, Justin was Group Financial Controller at Rubicor Group Ltd (ASX: RUB) one of the largest networks of specialist recruitment businesses in Australia. Justin holds a Bachelor of Business (Accounting) and is a member of Chartered Accountants Australia and New Zealand. Kim Stagg Chief Product and Technology Officer Kim Stagg Chief Product and Technology Officer, Appen Kim Stagg is the Chief Product and Technology Officer at Appen, bringing over 20 years of experience in AI-first strategies, B2B/B2C SaaS, and product leadership across a variety of industries. With deep expertise in product innovation, execution, and operational efficiencies, Kim has successfully led large-scale product transformations that have driven global business growth. Prior to Appen, Kim held senior leadership positions including Vice President of Product at Appen, Chief Technology Officer at Antea Group, and Chief Product Officer at iEHS. Kim holds a PhD in Hydrogeology and Computer Science from the University of Birmingham. Roc Tian SVP and General Manager, Appen China + JK Roc Tian SVP and General Manager, Appen China + JK As the Senior Vice President and General Manager for China, Japan, and Korea, Roc Tian is responsible for overseeing Appen's business strategy, sales, marketing, delivery, operations, and government relationships in these regions. He brings over 20 years of experience in sales, consulting, and management at Fortune 100 companies to his role at Appen. Roc’s invaluable expertise includes his previous role as Senior Partner at IBM GBS, where he led client services, public sector, and enterprise application service lines in China, Hong Kong, and Taiwan. Roc also played a pivotal role in growing IBM's global delivery center in China from 4,000 to over 10,000 employees. Prior to IBM, Roc served as a Business Quality Director at HP's global delivery center in China and was the founder and CTO of a successful tech startup. Roc holds a Master of Computer Applications and a PhD of Computer Software from Shanghai Jiaotong University. Sergio Bruccoleri Vice President, Delivery Sergio Bruccoleri Vice President, Delivery Sergio Bruccoleri is the VP of Delivery at Appen, bringing deep expertise in AI data operations, large-scale delivery, and technical execution across global programs. With more than a decade leading teams in the AI and digital transformation space, he has built and managed complex delivery ecosystems that support enterprise-grade AI development.Prior to Appen, he held senior leadership roles at Centific and Pactera, where he oversaw solution delivery, technical architecture, and global operations for high-impact AI data, localization, and platform modernization initiatives. Si Chen Vice President, Strategy and Marketing Si Chen Vice President of Strategy and Marketing Si is the VP, Strategy & Marketing at Appen. Si brings extensive experience in technology leadership roles and her areas of expertise include traditional AI/ML models, generative AI, multimodal AI systems, intelligent robotics, and AI industry solutions. Prior to joining Appen, Si held leadership positions as Head of Strategy, Partnerships & Operations at Tencent AI & Robotics Lab, and Head of Strategy & Business Development at AWS China. Si holds a Bachelor of Commerce from the University of New South Wales. Brian Jenkins Vice President, Sales Brian Jenkins Vice President, Sales With over 15 years of strategic selling experience, Brian Jenkins serves as Vice President of Sales across NA, LATAM and EMEA.  Prior to stepping into his current leadership role, Brian held key sales and growth positions at Scale AI and Snorkel, where he developed deep domain expertise in the data labeling and AI enablement space.  Earlier, Brian spent ~8 years at Salesforce, contributing to the company’s expansion across a multitude of verticals.Brian holds a Bachelors in Marketing from Clemson University along with a dual MBA from Coastal Carolina University and a FH-Mainz, University of Applied Sciences in Germany. Robert Page Vice President, Delivery for Strategic Accounts Francisco Rivera Vice President, Crowd Operations Francisco Rivera VP of Crowd Operations, Appen Francisco is the VP of Crowd Operations at Appen, bringing more than 20 years of experience leading operations, strategy, and growth teams across a diverse set of industries and markets. Prior to joining Appen, he spent more than 10 years driving operational excellence in leading digital marketplaces, such as Uber and Angi. Francisco holds a Bachelor of Science degree in Industrial Engineering from the University of Lima (Peru), as well as an MBA from Yale University. Jaime Frasca General Counsel and Company Secretary Jaime Frasca General Counsel and Company Secretary Jaime is the General Counsel at Appen and oversees the company’s legal, governance and risk functions. She brings over 20 years of experience advising global organisations across technology, consumer and media sectors, with deep expertise in regulatory and compliance matters.Prior to joining Appen, Jaime served as Senior Legal Director and Company Secretary (ANZ) at Dell Technologies. She began her career in private practice at leading Australian law firms before moving in-house to senior legal roles at Penguin Books UK and KFC South Pacific. She is admitted as a Solicitor of the Supreme Court of New South Wales and is a Graduate of the Australian Institute of Company Directors. Innovating for a better future Appen’s AI for good Discover how our AI innovations are driving positive change in the world and making a lasting impact, while staying true to our commitment to be good, do good, and lead good. Learn more Appen careers Join our global team of passionate and talented individuals who are working together to shape the future of AI. At Appen, you will have the opportunity to work on cutting-edge projects and collaborate with experts from around the world. Visit our careers page to explore current opportunities. Careers In the news Stay up to date on the latest news, updates, and announcements from Appen. Appen research identifies refusal as a safety strategy Appen drives Agentic AI innovation with custom data and end-to-end evaluation How Multimodal AI Will Spawn A New Wave Of Innovation Exclusive Interview: Appen CEO Ryan Kolln on Training the World's Most Powerful AI Models View more news Board of directors Vanessa Liu Non-executive Chair Vanessa has a deep expertise of emerging technology trends and enterprise uptake of artificial intelligence, especially in the US market. She is the Founder and CEO of SaaS technology company Sugarwork and is a non-executive director of Goodman Group (ASX: GMG). Most recently she was the Vice President of SAP.iO, the early-stage venture arm of SAP. Prior, Vanessa was the Chief Operating Officer at Trigger Media Group, a digital media incubator. Before that, Vanessa was Associate Partner at McKinsey & Company’s Media and Entertainment Practice where she served clients in media and high-tech sectors on issues of digital media strategy, emerging market strategy, growth and innovation. Vanessa graduated magna cum laude with an AB in psychology from Harvard University and cum laude with a JD from Harvard Law School. She serves as President of the Harvard Alumni Association. Vanessa is a member of:• People and Culture Committee• Audit and Risk Management Committee Ryan Kolln CEO and Managing Director Ryan Kolln is the Chief Executive Officer and Managing Director of Appen. Ryan brings over 20 years of global experience in technology and telecommunications, along with a deep understanding of Appen’s business and the AI industry. His professional career began as an engineer, with a focus on mobile network data engineering in Australia, Asia and North America. On completion of an MBA from New York University, Ryan joined The Boston Consulting Group (BCG) in 2011 as a strategy consultant. During his time at BCG he specialised in technology and telecommunications and gained deep strategy expertise across a variety of growth and operational topics. Steve Hasker Non-executive Director Steve has extensive experience as a CEO, COO and Advisor in the US. Since March 2022, he has held the position of President and CEO of Thomson Reuters. He was a Senior Advisor to private equity firm TPG Capital (2019–2020) and CEO of CAA Global, a TPG Capital portfolio company (2018–2019). At Nielsen Holdings PLC, Steve served as Global President and COO (2015–2017) and President, Global Products (2009–2014). Steve was a partner with McKinsey (1998–2009). Before that, Steve spent five years in several financial roles in the U.S. and other countries. He holds an MBA and a master’s degree in international affairs from Columbia University. Steve is Chair of the People and Culture Committee. Robin Low Non-executive Director Robin has extensive finance, risk and business experience from her 28-year career at PricewaterhouseCoopers where she was a partner specialising in assurance and risk, mainly in financial services. She was previously deputy chair of the Auditing and Assurance Standards Board. Robin is an experienced non-executive director, holding other current ASX directorships with IPH Limited (ASX: IPH) and Articore Limited (ASX:ATG). Her previous ASX board roles included CSG Limited (ASX: CSV), AUB Group (ASX: AUB) Limited and Marley Spoon SE (ASX: MMM). Robin is also a director of the Guide Dogs NSW/ACT and the Sax Institute, a member of Anacacia’s Business Advisory Council and she is a member of the Audit Committee for the University of NSW. Robin has a Bachelor of Commerce from the University of New South Wales and is a Fellow of the Institute of Chartered Accountants Australia and New Zealand and a Fellow of the Australian Institute of Company Directors. Robin is Chair of the Audit and Risk Management Committee. Stuart Davis Non-executive Director Stuart is a director of NEXTDC Limited (ASX: NXT) and Chair of the Remuneration Committee, a non-executive director of Bank of South Pacific Limited and PayPal Australia Ltd where he serves as Chair of the Risk Committee at both companies. He has more than 30 years’ experience as an international banker with the HSBC Group where he was CEO, HSBC India (2009–2012), CEO and Executive Director for HSBC Bank Australia Limited (2002–2009) and CEO HSBC Taiwan (1999–2002). He was a member of the Australian Bankers Association from 2002–2009 and Deputy Chair from 2006–2009. Stuart holds a LLB from Adelaide University and is a Graduate of the Australian Institute of Company Directors. Stuart is a member of the Audit and Risk Management Committee. Lynn Mickleburgh Non-executive Director Lynn has experience as an ASX non-executive director, a board advisor and transformational leader of both Fortune 500 companies and high growth SaaS companies. Lynn is a former non-executive Director of ASX listed Altium Limited (ASX:ALU), where she chaired the HR Committee and served on the Audit and Risk Committee. Previously, she was Head of Business Optimisation at Atlassian Inc, VP Finance at Citrix Systems and held various global and operational roles at Adobe Systems and Apple Computer. She holds a Bachelor of Science in Mathematics and an MBA in Business Management. Lynn is a member of the People and Culture Committee. Offices US Headquarters 4040 Lake Washington Blvd NE, Suite 320Kirkland, WA 98033 Australia Headquarters Level 6, 9 Help Street Chatswood NSW 2067 Exeter Rockeagle House Pynes Hill Exeter, EX2 5AZ, Devon Sydney 9 Help Street, Suite 6, NSW 2067 Shanghai Unit 01, 7th Floor, No. 6, Lane 2889, Jinke Road, China (Shanghai) Pilot Free Trade Zone Beijing Room 08B-18, Block D, 8th Floor, No. 28 Xinxi Road, Haidian District, Beijing Wuxi Room E406, 408, 410, 412, 401, 403, 405, 407, 409, 411, Liye Building, University Science Park, No. 20, Qingyuan Road, Xinwu District, Wuxi Dalian No. 1, 9th Floor, Unit 1, No. 555, Huangpu Road, Qixianling, Dalian High-tech Industrial Park, Liaoning Province Chongqing Building B09, No. 107, Data Valley Middle Road, Xiantao Street, Yubei District, Chongqing Huainan 4th Floor, Building 2, South Technology Incubator, Taifeng Street, West Huaihe Avenue, Shannan New District, Huainan City, Anhui Province Tokyo 9F 1-5-1 Shin Marunouchi Building, Marunouchi Chiyoda-ku Tokyo Japan, 100-6509 Cavite BPO building 1 Suntech iPark Lancaster New City, Barangay Alapan II Imus Cavite, Philippines Hyderabad Office 2, Eighth Floor, Tower 1, Sattva Knowledge Park, Raidurg, Hyderabad, Telangana, India - 500081 Hanoi 5F, Viet A Tower, No 9 Duy Tan street, Dich Vong Hau ward, Cau Giay district, Hanoi City, Viet Nam High quality data at scale Discover how Appen accelerates the development of your AI applications. Schedule a demo Contact us Thank you for getting in touch! We appreciate you contacting Appen. One of our colleagues will get back in touch with you soon! Have a great day! --- Beyond the leaderboard. See how real AI performance is measured Collaborating with Appen's valued partners, we empower organizations to unlock greater value, seize new business opportunities, and foster unwavering customer loyalty and trust. Get started Learn more about our partnerships program. Schedule a demo Contact us Thank you for getting in touch! We appreciate you contacting Appen. One of our colleagues will get back in touch with you soon! Have a great day! --- Beyond the leaderboard. See how real AI performance is measured Back Webinars Beyond the Leaderboard: Bridging Research and Real-World AI Performance Nov 11, 2025 • On-Demand • 1 hour Watch now On-demand webinar Read the transcript On-Demand Webinar Overview Public benchmarks help us understand progress in AI but they rarely reflect real-world performance. In this session, SAP’s Chief Data Scientist, Daniel Dahlmeier, breaks down why enterprise teams see such a gap between leaderboard results and deployed model behaviour, and what it takes to evaluate models in a way that truly represents real-world outcomes. This webinar covers practical, research-backed techniques to measure accuracy, safety, and reasoning more effectively across LLMs, multimodal models, and agents. What You’ll Learn Why benchmark results often fail in production‍Static prompts, model-centric evaluation, and narrow test sets can misrepresent true rankings. How SAP approaches enterprise-grade evaluations‍Including domain-specific datasets, multimodal document extraction, prompt optimization, and safety assessments. The impact of prompt optimization on model rankings‍SAP’s research shows that when prompts are optimized per model, leaderboard positions shift dramatically, sometimes flipping top and bottom performers. The future of safety evaluation & automated red teaming‍Why static harmful-prompt benchmarks fall short, and how dynamic red teaming uncovers deeper vulnerabilities. How enterprises should evaluate agents‍From static to dynamic multi-turn evaluation, tool-use assessment, and measuring partial progress to understand where agents fail and why. Who This Is For AI leaders and product owners building enterprise AI systems ML engineers and researchers running LLM evaluations Teams deploying agents, copilots, or automation workflows Anyone wanting a practical, realistic view of model performance Speakers Daniel Dahlmeier Chief Data Scientist SAP Daniel is a Data Science Manager and the Chief Data Scientist at the SAP Business AI unit, where his team develops deep-learning models for document processing and benchmarking. He also serves as an adjunct assistant professor at the Singapore University of Technology and Design and teaches at Heidelberg University. Si Chen VP, Strategy & Marketing Appen Si leads strategy and marketing at Appen and brings extensive experience across traditional AI/ML models, generative AI, multimodal AI systems, and intelligent robotics. Prior to Appen, she held leadership positions at Tencent AI & Robotics Lab and AWS China, driving innovation and partnerships in applied AI. Talk to our team