名前

名前よみがな

携帯キャリアメールは非推奨です。携帯キャリアメールを登録した場合、通知メール等が迷惑メールと誤認されて届かない可能性があります。

メールアドレス

現所属

年齢

最終学歴

経歴

ポートフォリオサイトやSNSアカウントなど、ご自身に関するWebサイトのURLをご記載ください。

リンク

現年収

希望年収

応募先へのメッセージ

履歴書

職務経歴書

その他のファイル

現在、転職をお考えの時期感についてご教示ください。
・1〜3ヶ月以内に転職したい
・半年以内に転職を考えている
・その他

転職のご意向について

差し支えなければ、転職を検討されている理由を簡単にご記入ください。

転職をお考えの理由について

例：SNS、note、新聞記事、知人の紹介 など

差し支えなければ、Rechoを知っていただいたきっかけをご教示ください

Senior AI Inference Engineer (LLM & Speech Models)

## In short

You'll design and build Recho's inference framework for speech models and LLMs, and you'll own its throughput and GPU cost. This is a hands-on senior role: you'll architect the system and write much of the code yourself.

- **Team language:** English (Japanese not required)
- **Salary:** ¥10M–¥20M per year, plus stock options
- **Location:** Tokyo (Hanzomon), hybrid. Candidates based outside Japan are welcome. We offer visa sponsorship and relocation support, and we're open to discussing fully remote arrangements.

## You might be a fit if you are…

- An MLOps / ML platform engineer who has built and tuned LLM inference stacks (vLLM, SGLang, etc.)
- A speech recognition or speech synthesis engineer/researcher who wants to shift toward inference systems
- An engineer who has optimized throughput and cost for inference workloads on large GPU clusters

## About the role

At Recho, we run training, evaluation, data generation and large-scale batch inference across multiple speech models and LLMs. Your core mission is to independently design and implement a high-efficiency inference framework that serves all of them.

This isn't about wrapping a model in an API or tweaking config files. You'll need to understand each model's architecture, inputs and outputs, sequence lengths, pre- and post-processing, and GPU characteristics. From there, you'll separate and integrate model execution, batching, data processing and scheduling into a reusable inference architecture.

On the LLM side, you'll build on mature engines such as vLLM and SGLang. You'll tune model adaptation, inference parameters, batching, prefill/decode behavior and request priority. Across both speech and LLMs, your north star is throughput and GPU cost.

## What this role is *not*

- Not an on-call or SRE role. SLA management, incident response and infrastructure maintenance for online services are not the focus.
- GPU cluster operations and general DevOps are handled by a separate team.
- Writing CUDA kernels or building an inference engine from scratch is not the main job.
- RL training infrastructure belongs to our separate RL Infrastructure Engineer role. You'll partner with that team on the inference side.

## What you'll do

### 1. Build a common inference framework for speech models

- Design a shared inference architecture for ASR, TTS, speech-to-speech and other audio generation models
- Draw clear boundaries between shared and model-specific modules, with adapter/plugin interfaces for onboarding new models
- Build the inference APIs, SDKs and tooling our model teams use every day

### 2. Optimize large-scale batch inference for audio

- Apply dynamic batching and bucketing for variable-length audio, and run data-parallel inference across GPUs and nodes
- Build asynchronous pipelines that overlap preprocessing, inference, postprocessing and storage I/O
- Measure and remove bottlenecks based on end-to-end processing time and cost

### 3. Adapt and optimize LLM inference

- Stand up LLM inference on mature engines such as vLLM and SGLang
- Tune tensor/pipeline parallelism, continuous batching, KV cache and prefix cache
- Design scheduling that accounts for prefill/decode characteristics and task priority

### 4. Plan GPU usage for inference workloads

- Handle GPU allocation and capacity planning for batch inference and evaluation
- Orchestrate GPU inference jobs with Kubernetes, Slurm and Ray
- Track and reduce GPU usage and inference cost per model and per task

We're happy to go into more detail during the intro call and interviews.

## Why this role

- **Set the playbook in an area that doesn't have one yet.** Inference for variable-length audio, codecs, vocoders, speech-to-speech and multimodal models has no established best practices. What you design becomes the foundation for every future model and team.
- **Own inference end to end.** You'll cover both speech and LLMs, from framework design to batch inference and scheduling. That means you optimize the whole system, not just one piece of it.
- **Directly move the business.** Inference efficiency determines GPU cost per unit of generated speech and response latency. Those decide how competitive our product is for the major enterprises running it in production.

## Tech stack

- **Languages & frameworks:** Python, PyTorch
- **LLM inference:** vLLM, SGLang, TensorRT-LLM, Triton Inference Server
- **Speech models:** ASR, TTS, speech-to-speech, audio codecs, vocoders
- **Orchestration:** Kubernetes, Slurm, Ray
- **Low-level optimization (nice to have):** C++, CUDA, Triton, NCCL

## Our product

We build the Recho AI Voice Platform, which delivers voice conversations natural enough to be hard to tell apart from a human. We also deploy and operate Voice AI Agents for large enterprises.

Our edge is a platform for building and improving high-quality agents quickly. It captures industry know-how and turns user feedback into a fast, continuous improvement loop. Our agents now handle complex, large-scale contact center operations in finance, manufacturing and the public sector. In some deployments, they've resolved every call without handing off to a human agent.

## Hiring process

Intro call → Case assignment & first interview → Paid work trial* → Final interview → Offer meeting

\*The work trial is 1–2 consecutive days at our Tokyo office, working on real tasks alongside the team, and it's paid. It's a chance for both sides to check fit and get a feel for the work.

- **If you're based outside Japan,** we'll agree on the format together. We won't ask you to travel to Japan just for this step.
- If your schedule or your current employer's side-work policy makes the trial difficult, just let us know.
- We may also ask for references.

## About Recho

Recho develops foundation models for customer-support AI agents and also deploys them for clients, all within one organization. Our core strength is voice AI technology we built entirely in-house, which resolves the industry's long-standing trade-off between quality and cost.

We've closed a Series A of approximately ¥700M and are rapidly rolling out to demanding production environments at major companies such as SBI Shinsei Bank and KDDI.

## Learn more

- [Corporate website](https://recho-ai.com/)
- [Team interviews (note, in Japanese)](https://note.com/recho_ai)
- [Company deck](https://recruit.recho-ai.com/)
- [Case study: Partnership with KDDI and Altius Link (Japanese)](https://prtimes.jp/main/html/rd/p/000000021.000159800.html)
- [Case study: SBI Shinsei Bank (Japanese)](https://mainichi.jp/articles/20260130/k00/00m/020/275000c)
- [2-minute voice demo](https://recruit.recho-ai.com/#2f276fb799f480d5bba7d5784d6ad31e) — give it a listen!

Senior AI Inference Engineer (LLM & Speech Models) 応募フォーム