Joe DeGregorio

Principal-level applied AI and data science leader

I help teams turn ambiguous AI opportunities into systems with real business value.

I work at the intersection of product, analytics, and engineering, especially when the path is unclear and a new opportunity has to be created from scratch. My focus is agentic systems, evaluation, and measurement.

Joe DeGregorio portrait

Expertise

How I work

I’m strongest when technical possibility needs to become something useful, measurable, and durable.

01

Find the leverage

Start with the real operating problem, the key constraint, and the smallest intervention that can change the outcome.

02

Build the first proof

Use focused prototypes to create clarity, alignment, and momentum instead of waiting for the full plan.

03

Measure what matters

Define evaluation, instrumentation, and feedback loops early so teams know what to trust and how to improve it.

What I do

The focus stays practical: clear workflow design, clear measurement, and clear proof that the system is doing something worthwhile.

AI/ML workflows

Design agentic and decision-support systems that fit a real operating context, not just a demo.

Evals and measurement

Build quality models, scorecards, and instrumentation that make progress legible.

Prototype strategy

Use focused prototype wedges to de-risk bigger product, architecture, and organizational bets.

Selected work

Proof of value beyond what fits on a resume.

These summaries go one level deeper: the problem, the system, and the value created. Expand any row to see the work in context.

Flagship system · 2024

Automating knowledge creation to strengthen downstream RAG agents

Built a fully automated, end-to-end agentic workflow that drafts, reviews, revises, and improves knowledge content so the RAG agents built on top of it have better material to retrieve.

Challenge

Knowledge creation could not keep pace with product change, which meant downstream self-help and support agents were limited by stale or uneven source content.

Value created

Turned knowledge creation into a repeatable system, improved draft quality and throughput, and created built-in feedback signals that helped both the content pipeline and the complementary RAG agents improve over time.

Expand

Role

Builder-strategist shaping the workflow, the quality model, and the feedback design that made the automation trustworthy.

System

Orchestrated a multi-agent pipeline with retrieval, drafting, groundedness review, style review, and automated revision loops before publication or human signoff.

What was distinctive

End-to-end agentic workflow rather than a drafting assistant bolted onto manual steps.
Automated groundedness and style-review sub-agents fed revision loops instead of producing one-shot output.
Better source content strengthened the downstream support and self-help agents that relied on retrieval.

Flagship system · 2025

Designing measurement systems for AI support workflows

A public-safe look at agent performance measurement, hybrid evaluation approaches, and the realities of quality signals in production-like support systems.

Challenge

Support teams needed a way to understand whether AI-assisted workflows were actually helping, not just producing activity.

Value created

Created a clearer quality bar and a more trustworthy way to compare workflow variants before broader rollout, shifting the conversation from anecdotes to evidence.

Expand

Role

Principal-level applied AI and data science partner shaping the problem framing, evaluation model, and instrumentation plan.

System

Built a layered quality model that combined rubric-based review, operational signals, feedback loops, and experiment-ready instrumentation.

What was distinctive

Separated convenience metrics from genuine quality signals.
Designed for human review and behavioral signals to work together instead of forcing one source of truth.
Made later experimentation possible by instrumenting the workflow early.

Flagship system · 2024

Self-help, search, and reducing support friction

Combining analytics, instrumentation, experimentation, and systems thinking to improve discoverability and customer outcomes.

Challenge

Customers had self-help options available, but discoverability gaps and workflow friction kept pushing demand into expensive support paths.

Value created

Surfaced where customer intent, search behavior, and support outcomes diverged most sharply so teams could prioritize the changes that reduced friction fastest.

Expand

Role

Cross-functional problem framer connecting analytics, product questions, and experience design into one measurement-backed workflow.

System

Mapped the full self-help journey, instrumented search and support touchpoints, and used experiments to identify the highest-leverage improvements.

What was distinctive

Reframed the problem from search quality alone to an end-to-end guidance and resolution system.
Used instrumentation and experimentation together rather than treating analytics as a reporting layer.
Connected user intent to real support outcomes so the work stayed grounded in customer value.

Foundation · 2021

Predictive analytics for warranty and operational decisions

Earlier-career work that shaped my approach to high-value modeling, risk prioritization, and operational impact.

Challenge

Operational and warranty decisions involved uncertainty, long feedback cycles, and meaningful business cost.

Value created

Improved risk visibility and sharpened how business partners thought about prioritization under uncertainty, with direct relevance to planning and resource allocation.

Expand

Role

Data scientist translating operational questions into models and decision support that stakeholders could actually use.

System

Combined predictive modeling, domain understanding, and practical interpretation so the output fit the decisions teams needed to make.

What was distinctive

Grounded modeling work in operational decisions instead of abstract accuracy alone.
Balanced technical rigor with explainability for business stakeholders.
Built the analytic backbone that now shows up in how I design AI systems: measurable, practical, and decision-linked.

Experience

A fuller view of the arc.

My career moved from engineering systems to analytics and predictive modeling to applied AI. The through-line has stayed the same: define the problem well, build something useful, and make the result measurable.

Open resume PDF

Oct 2024 - Present

AI/ML Engineer (P60)

Atlassian

What I did

Leading applied AI work across support, knowledge, and service systems, from workflow design through evaluation and instrumentation.

Key results

Pushed deeper on agentic systems, content workflows, and the operating models needed to improve quality over time.

What it shaped

Sharpened how I balance principal-level judgment with hands-on building, especially around trust, measurement, and system boundaries.

Oct 2023 - Sep 2024

Principal Data Scientist (P50)

Atlassian

What I did

Set direction for AI customer-support strategy, experiment design, and cross-functional execution during the first wave of GPT-driven opportunity.

Key results

Helped move support AI from exploration into measurable production work, including meaningful efficiency gains and patentable technical approaches.

What it shaped

Made it even clearer that the hard part of AI work is not model novelty; it is turning possibility into an operational system teams can trust.

Feb 2022 - Sep 2023

Senior Data Scientist (P40)

Atlassian

What I did

Worked at the boundary of customer support, product analytics, and early AI adoption, connecting business questions to workable system designs.

Key results

Improved support efficiency, modernized chat and self-help experiences, and built credibility for broader AI investment.

What it shaped

This was the inflection point where my analytics background fused with product-systems thinking and applied AI.

Mar 2019 - Feb 2022

Data Scientist

PACCAR

What I did

Led predictive modeling and decision-support work across warranty, used trucks, and planning while defining an MLOps roadmap.

Key results

Delivered multi-million-dollar value across warranty optimization, sales opportunity sizing, and forecast-driven planning.

What it shaped

Reinforced my preference for data science that changes operational decisions, not just dashboards.

Nov 2017 - Feb 2019

Senior Analyst (Warranty)

PACCAR

What I did

Bridged warranty business context and analytics, building early NLP and survival-analysis approaches for claims and recall prioritization.

Key results

Improved the ability to spot high-risk vehicles, detect emerging claim patterns, and focus interventions sooner.

What it shaped

Built the habit of working backward from high-value decisions rather than isolated metrics.

Sep 2016 - Nov 2017

Senior Systems Engineer - Vehicle Integration

PACCAR

What I did

Owned functional requirements and systems work for vehicle software and other integration-heavy programs.

Key results

Tightened requirements quality and improved how complex cross-system work got managed.

What it shaped

Gave me a durable systems lens that still shows up in how I think about architecture, interfaces, and failure modes.

Jan 2016 - Aug 2016

Senior Engineer - Special Assignment

Caterpillar

What I did

Took on cross-functional engineering assignments in a high-stakes product environment where product issues and operational needs intersected.

Key results

Helped close product and quality gaps with pragmatic engineering follow-through.

What it shaped

Strengthened my comfort stepping into ambiguous work and creating structure quickly.

Oct 2012 - Dec 2015

Design Engineer

Caterpillar

What I did

Worked on drivetrain and transmission-related engineering, product quality, and production-facing problem solving.

Key results

Reduced warranty and scrap costs while improving test and quality processes.

What it shaped

Grounded my approach in real-world systems where reliability and cost both matter.

Jun 2011 - Sep 2012

Engineer - Leadership & Technology Development Program

Caterpillar

What I did

Rotated through early-career engineering assignments and learned how product, manufacturing, and program execution fit together.

Key results

Built range quickly by working across disciplines rather than inside one narrow lane.

What it shaped

Established the habit of learning fast in unfamiliar contexts.

May 2010 - Aug 2010

Project Engineer Intern

Centro, Inc.

What I did

Supported project engineering work in a manufacturing setting early in my career.

Key results

Got a first view of how engineering decisions translate into production realities.

What it shaped

Made hands-on, operationally grounded work feel like home.

Contact

Interested in applied AI, product systems, or technical leadership?

I’m always happy to talk about applied AI, evaluation, product systems, and principal-level technical leadership in high-ambiguity environments.