Data Scientist, Trust & Safety

Replit5 days ago

Location

Foster City, CA

Type

Full Time

Salary

USD 210,000 – 310,000

Level

Senior

Role

Data Engineer

Posted

Jul 20, 2026

Full TimeSenior

The role

Summary

Replit seeks a Data Scientist for Trust & Safety to build abuse detection and fraud prevention systems from the ground up on an AI-native platform. You'll develop measurement frameworks, predictive risk models, and anomaly detection systems to protect millions of users while balancing security with legitimate user experience. This role requires 5+ years of data science experience with strong SQL/Python skills, expertise in handling imperfect labels and high-stakes decisions, and the ability to communicate complex tradeoffs to cross-functional teams.

What you'll do

Own Trust & Safety Analytics Foundation: Establish and maintain the analytical foundation for Trust & Safety metrics including abuse prevalence, fraud loss, false-positive and false-negative rates, time to detect, time to mitigate, appeal and reversal rates, and verification step-up conversion. Create reliable measurement systems that serve as the single source of truth for abuse tracking across the platform.

Build Data Models and Pipelines: Design and implement robust dbt models and reliable datasets that integrate product events, account and identity signals, payment activity, infrastructure usage, content classifications, enforcement actions, appeals, and support outcomes. Ensure data quality and consistency across multiple source systems serving the Trust & Safety function.

Develop Risk and Detection Models: Create and evaluate predictive models, rules-based systems, and anomaly-detection algorithms targeting diverse threats including phishing, scam hosting, cryptomining, LLM token farming, payment fraud, promotional abuse, and AI-agent exploitation. Apply machine learning techniques including classification, anomaly detection, and risk scoring to identify adversarial behavior patterns.

Design and Execute Rigorous Experiments: Develop and conduct offline evaluations, shadow-mode tests, holdout analyses, and controlled experiments to measure detection quality and quantify user impact of new policies, enforcement actions, and progressive verification strategies. Establish rigorous statistical frameworks for causal inference that account for confounding variables and selection bias.

Define Thresholds and Decision Frameworks: Establish decision thresholds and frameworks that optimize the tradeoff between abuse reduction, economic loss, customer friction, and false positives across free, paid, and enterprise user segments. Communicate these tradeoffs clearly to enable informed decision-making by product and legal teams.

Investigate Emerging Abuse Patterns: Analyze emerging abuse patterns, quantify their business impact, identify coordinated behavior networks, and translate ambiguous signals into actionable recommendations for product and engineering teams. Combine behavioral signals with graph analysis to detect sophisticated, multi-layered attacks.

Build Predictive Risk Models: Develop predictive models that estimate account, device, transaction, workspace, or deployment risk. Embed these risk signals into detection workflows, human review queues, and escalation paths to enable proactive prevention and targeted intervention strategies.

Partner on Case Review and Appeals: Collaborate with Support and Legal teams to improve case review quality, appeals processes, reason-code taxonomy, and feedback loops. Transform human enforcement decisions into model signals and policy insights that continuously improve system performance.

Implement Monitoring and Drift Detection: Build comprehensive monitoring systems that detect model drift, attacker adaptation, data-quality failures, and unexpected harm to legitimate users. Establish alerts and playbooks for rapid response to model degradation or systematic bias in enforcement decisions.

Communicate Findings and Tradeoffs: Present analytical findings clearly to technical and non-technical stakeholders including executives, legal, support, and engineering teams. Articulate tradeoffs, quantify uncertainty, and provide evidence-based recommendations that enable high-impact decisions in complex abuse scenarios.

Leverage AI Tools for Efficiency: Use AI agents and analytical tools to accelerate analysis, hypothesis generation, code development, and investigation prototyping. Maintain high analytical standards by treating all AI-assisted output as draft work requiring rigorous validation before delivery.

Measure False Positives and User Harm: Quantify false-positive rates, appeal reversal patterns, and enforcement harm across user segments. Design and evaluate human-review workflows, feedback mechanisms, and progressive verification ladders that balance abuse prevention with legitimate user experience and conversion.

Support Graph Analysis and Entity Resolution: Apply graph analysis, entity resolution, and coordinated-behavior detection techniques to identify attack clusters, fraud rings, and organized abuse networks. Combine identity, device, payment, and behavioral signals to surface sophisticated multi-account exploitation patterns.

Evaluate Anti-Abuse Policies: Design rigorous policy evaluation frameworks using counterfactual analysis methods such as difference-in-differences, propensity score matching, synthetic control, and uplift modeling. Estimate policy impact before launch and recommend whether to proceed, revise, or reject based on quantified evidence.

Build Investigation and Detection Automation: Develop AI-powered analytical tools, automated investigation systems, and novel measurement approaches that scale human expertise. Create practical playbooks, triage queues, and escalation paths that operationalize analytical insights for review teams.

Address AI-Native Abuse Threats: Identify and mitigate abuse patterns specific to AI-native platforms including prompt injection attacks, LLM token farming, model extraction attempts, and agent-driven exploitation. Develop detection strategies tailored to the unique attack surface of algorithmic code generation platforms.

What we look for

Technical

SQL ProficiencyAdvanced SQL skills for writing complex queries against large behavioral datasets, including window functions, recursive queries, and optimization for analytical workloads. Ability to work efficiently with multi-billion-row tables and write maintainable, reusable SQL for production analytics.

Python ProgrammingStrong Python skills for data manipulation, statistical analysis, model development, and data pipeline implementation. Experience with pandas, scikit-learn, numpy, and other standard data science libraries. Ability to write production-quality, testable code.

Predictive Modeling and Machine LearningProven experience developing and evaluating predictive models including classification, anomaly detection, and risk scoring systems. Strong understanding of model evaluation metrics, feature engineering, hyperparameter tuning, threshold selection, and production model deployment.

Statistical Analysis and Experimental DesignDeep expertise in statistical methods including hypothesis testing, confidence intervals, and causal inference. Experience designing and analyzing A/B tests, controlled experiments, and observational studies with attention to confounding, selection bias, and multiple comparison correction.

Data Modeling and ETLExperience building reliable data models and ETL pipelines that power analytics and ML systems. Familiarity with data warehousing concepts, dimensional modeling, and tools for orchestrating data transformations. Understanding of data quality validation and monitoring.

Handling Imperfect DataDemonstrated ability to work with incomplete labels, biased samples, delayed ground truth, and noisy signals. Skill in quantifying uncertainty, understanding sources of bias, and making sound decisions despite data limitations. Experience in domains with high-stakes decisions and real consequences.

Fraud and Risk AnalysisUnderstanding of fraud detection methodologies, risk modeling, account takeover prevention, and payment security. Knowledge of common fraud tactics, detection evasion techniques, and the economic incentives driving adversarial behavior on digital platforms.

Large Behavioral DatasetsExperience working with large-scale event streams, user behavior logs, and transactional data. Ability to aggregate, join, and analyze billions of events to extract meaningful insights. Familiarity with handling data sparsity and long-tail phenomena in user behavior.

Model Monitoring and Drift DetectionExperience implementing production monitoring for ML models including performance tracking, data drift detection, prediction drift identification, and alert systems. Understanding of model degradation causes and strategies for rapid retraining and rollback.

Education

Bachelor's Degree in Quantitative FieldDegree in Computer Science, Mathematics, Statistics, Engineering, Economics, Physics, or related quantitative discipline. Strong foundation in mathematics, statistics, and scientific method principles.

Experience

5+ Years Data Science or Related FieldMinimum 5+ years of professional experience in data science, product analytics, fraud prevention, risk management, trust and safety, security analytics, or a closely related field. Demonstrated track record of delivering analytical projects that impact business outcomes.

Fraud or Anti-Abuse SystemsPreferred experience building or evaluating anti-abuse, fraud detection, identity verification, security, spam filtering, integrity enforcement, or content-safety systems at significant scale. Understanding of adversarial dynamics, attacker adaptation, and the operational challenges of enforcement.

Production ML Model DeploymentHands-on experience shipping and maintaining ML models in production environments including classification systems, anomaly detectors, and risk scorers. Experience with feature engineering on behavioral and transaction data, threshold calibration against precision-recall economics, and post-launch monitoring.

Graph Analysis and Coordination DetectionExperience with graph analysis, entity resolution, coordinated-behavior detection, reputation systems, or anomaly detection on networked data. Ability to identify clusters, rings, and organized networks of coordinated accounts from sparse or noisy signals.

False Positive Measurement and Harm AnalysisExperience measuring and minimizing false positives, quantifying enforcement harm, designing human-review workflows, appeals processes, or using case outcomes and appeals data as model feedback. Understanding of the user experience impact of incorrect enforcement decisions.

Consumer Platform or Adversarial SurfaceExperience at a consumer-facing platform, developer tool, cloud provider, marketplace, fintech company, or other product with meaningful adversarial dynamics. Understanding of freemium, usage-based, or promotional pricing models and the abuse incentives they create.

Modern Data StackPractical experience with modern data infrastructure including dbt, BigQuery, Snowflake, Fivetran, Amplitude, Mixpanel, or Segment. Familiarity with cloud data warehousing, event tracking systems, and analytics engineering practices.

Skills

Required skills

SQL and Data QueryingExpert-level SQL for complex analytical queries, data exploration, and model feature engineering on large datasets. Proficiency with window functions, CTEs, query optimization, and writing maintainable analytical SQL.

Python Data ScienceStrong Python programming for statistical analysis, model development, and data pipeline creation. Expertise with pandas, numpy, scikit-learn, and ability to write production-quality, tested code.

Predictive ModelingExperience developing, evaluating, and deploying classification models, regression models, anomaly detectors, and risk-scoring systems. Understanding of train/test splits, cross-validation, feature selection, and threshold optimization.

Statistical Rigor and ExperimentationAdvanced statistical knowledge including hypothesis testing, confidence intervals, p-values, effect sizes, and multiple comparison correction. Experience designing and analyzing controlled experiments with attention to confounding and selection bias.

Data Analysis and Insight TranslationAbility to translate data analysis into clear, actionable recommendations. Skill in identifying spurious correlations, pressure-testing findings, quantifying uncertainty, and communicating complex tradeoffs to diverse audiences.

Handling Imperfect InformationDemonstrated ability to work with incomplete labels, biased samples, and high-stakes decisions. Comfort with ambiguity, delayed ground truth, and noisy signals. Strong judgment about when data is sufficient for decision-making.

Causal ThinkingUnderstanding of causal inference principles, confounding variables, selection bias, and treatment effect estimation. Ability to distinguish correlation from causation and design analyses that support causal claims.

Adversarial MindsetAbility to think like an attacker and reason about adversarial incentives, evasion techniques, and attacker adaptation. Skill in designing detections that account for intelligent adversaries who react to enforcement.

AI Tool UsageProficiency using AI agents, copilots, and LLM-based tools for code generation, data exploration, and hypothesis generation. Discipline to treat all AI-assisted outputs as drafts requiring human validation before delivery.

Nice to have

Graph Analysis and Network DetectionExperience with graph algorithms, entity resolution, community detection, and coordinated-behavior identification. Ability to identify fraud rings, attack clusters, and organized networks from sparse signals on heterogeneous networks.

Fraud Domain ExpertiseDeep knowledge of fraud detection, payment fraud, identity fraud, account takeover, and financial crime patterns. Understanding of fraud prevention controls, regulatory requirements, and the adversarial cat-and-mouse dynamics of fraud prevention.

Trust and Safety OperationsExperience with trust and safety systems, content moderation, abuse prevention, or platform integrity enforcement. Knowledge of how automated systems, human review, and appeals processes interact to reduce abuse while maintaining user trust.

dbt and Modern Data StackHands-on experience with dbt for building maintainable, version-controlled data models. Familiarity with cloud data warehouses like BigQuery or Snowflake, and analytics platforms like Amplitude or Mixpanel.

Progressive Verification and IdentityKnowledge of progressive verification strategies, KYC/AML procedures, account trust scoring, and identity verification providers such as Prove, Persona, Socure, or Stripe Identity. Understanding of the user experience tradeoffs in identity verification.

Causal Inference MethodsAdvanced knowledge of causal inference techniques including difference-in-differences, propensity score matching, synthetic control, instrumental variables, and uplift modeling. Ability to estimate treatment effects from observational data.

Production ML PipelinesExperience building, deploying, monitoring, and maintaining ML models in production including feature stores, model serving, retraining pipelines, and fallback strategies. Understanding of model governance, versioning, and reproducibility.

AI-Native Abuse PatternsUnderstanding of AI-specific abuse including prompt injection, LLM token farming, model extraction, API abuse, and agent-driven exploitation. Familiarity with unique attack vectors on AI-native platforms and AI-powered abuse detection.

Rapid Prototyping and IterationAbility to build and test hypotheses quickly using AI-assisted development and analytical tools. Skill in moving from ambiguous problem statements to concrete recommendations in short timeframes while maintaining analytical rigor.

Technical CommunicationExcellent ability to explain complex statistical and technical concepts to non-technical audiences including legal, support, and executive teams. Skill in translating analytical findings into practical decision frameworks and operational playbooks.