# User Data Export Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build a user-owned export pipeline that produces a versioned `manifest + data JSON + files` package with configurable masking and secure download. **Architecture:** Add an async export job table and service pipeline in Nitro server: request API creates job, worker executes export with a fixed cutoff timestamp, and download API serves completed artifacts. Export logic is split into small modules for domain data extraction, masking/sanitization, file collection, checksum generation, and manifest assembly. UI exposes a simple “Data Export Center” under `/me` for request, progress, and download. **Tech Stack:** Nuxt 4, Nitro server routes/tasks, Bun test, Drizzle ORM (sqlite schema + migrations), Node fs/crypto/path APIs, Vue + Nuxt UI. --- ## File Structure (planned changes) - Create: `packages/drizzle-pkg/database/sqlite/schema/export.ts`(导出任务表) - Modify: `packages/drizzle-pkg/database/sqlite/schema/index.ts`(导出新 schema) - Modify: `packages/drizzle-pkg/lib/schema/content.ts`(按现有导出方式补 export schema) - Create: `packages/drizzle-pkg/migrations/0010_user_export_tasks.sql` - Create: `server/constants/export.ts`(状态、策略、路径、TTL) - Create: `server/utils/export-mask.ts`(脱敏与永不导出过滤) - Create: `server/utils/export-hash.ts`(SHA256 计算) - Create: `server/service/export/jobs.ts`(任务创建/查询/状态机) - Create: `server/service/export/build-data.ts`(业务数据抽取) - Create: `server/service/export/build-files.ts`(媒体文件收集) - Create: `server/service/export/build-manifest.ts`(manifest 组装) - Create: `server/service/export/run.ts`(任务执行主流程) - Create: `server/tasks/export/process.ts`(定时/触发执行任务) - Create: `server/api/me/export/request.post.ts` - Create: `server/api/me/export/tasks.get.ts` - Create: `server/api/me/export/tasks/[id]/download.get.ts` - Create: `server/utils/me-export-request-body.ts`(请求参数校验) - Create: `server/utils/me-export-request-body.test.ts` - Create: `server/service/export/jobs.test.ts` - Create: `server/service/export/build-manifest.test.ts` - Modify: `app/pages/me/index.vue`(加入“数据导出中心”入口) - Create: `app/pages/me/export/index.vue`(导出页面) - Create: `app/types/export.ts`(前端 DTO) --- ### Task 1: Export Job Schema + Migration **Files:** - Create: `packages/drizzle-pkg/database/sqlite/schema/export.ts` - Modify: `packages/drizzle-pkg/database/sqlite/schema/index.ts` - Modify: `packages/drizzle-pkg/lib/schema/content.ts` - Create: `packages/drizzle-pkg/migrations/0010_user_export_tasks.sql` - Test: `bun run db:migrate` - [ ] **Step 1: Write the failing schema export test (smoke)** ```ts // File: server/service/export/jobs.test.ts (temporary first test) import { describe, expect, test } from "bun:test"; import { userExportTasks } from "drizzle-pkg/lib/schema/export"; describe("userExportTasks schema", () => { test("exposes required columns", () => { expect(userExportTasks.userId.name).toBe("user_id"); expect(userExportTasks.status.name).toBe("status"); expect(userExportTasks.maskPolicy.name).toBe("mask_policy"); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `bun test server/service/export/jobs.test.ts` Expected: FAIL with module or symbol not found for `drizzle-pkg/lib/schema/export`. - [ ] **Step 3: Add minimal schema + migration** ```ts // File: packages/drizzle-pkg/database/sqlite/schema/export.ts import { index, integer, sqliteTable, text } from "drizzle-orm/sqlite-core"; import { users } from "./auth"; export const userExportTasks = sqliteTable( "user_export_tasks", { id: integer().primaryKey(), userId: integer("user_id").notNull().references(() => users.id, { onDelete: "cascade" }), status: text().notNull().default("queued"), // queued|running|succeeded|failed|expired maskPolicy: text("mask_policy").notNull().default("masked"), // masked|raw exportCutoffAt: integer("export_cutoff_at", { mode: "timestamp_ms" }), outputDir: text("output_dir"), outputName: text("output_name"), totalBytes: integer("total_bytes"), errorCode: text("error_code"), errorMessage: text("error_message"), expiresAt: integer("expires_at", { mode: "timestamp_ms" }), createdAt: integer("created_at", { mode: "timestamp_ms" }).defaultNow().notNull(), updatedAt: integer("updated_at", { mode: "timestamp_ms" }).defaultNow().$onUpdate(() => new Date()).notNull(), }, (table) => [ index("user_export_tasks_user_id_idx").on(table.userId), index("user_export_tasks_status_idx").on(table.status), ], ); ``` ```sql -- File: packages/drizzle-pkg/migrations/0010_user_export_tasks.sql CREATE TABLE user_export_tasks ( id INTEGER PRIMARY KEY NOT NULL, user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, status TEXT NOT NULL DEFAULT 'queued', mask_policy TEXT NOT NULL DEFAULT 'masked', export_cutoff_at INTEGER, output_dir TEXT, output_name TEXT, total_bytes INTEGER, error_code TEXT, error_message TEXT, expires_at INTEGER, created_at INTEGER NOT NULL DEFAULT (unixepoch('subsec') * 1000), updated_at INTEGER NOT NULL DEFAULT (unixepoch('subsec') * 1000) ); CREATE INDEX user_export_tasks_user_id_idx ON user_export_tasks(user_id); CREATE INDEX user_export_tasks_status_idx ON user_export_tasks(status); ``` - [ ] **Step 4: Run tests and migration** Run: `bun test server/service/export/jobs.test.ts && bun run db:migrate` Expected: PASS test; migration applies without SQL error. - [ ] **Step 5: Commit** ```bash git add packages/drizzle-pkg/database/sqlite/schema/export.ts \ packages/drizzle-pkg/database/sqlite/schema/index.ts \ packages/drizzle-pkg/lib/schema/content.ts \ packages/drizzle-pkg/migrations/0010_user_export_tasks.sql \ server/service/export/jobs.test.ts git commit -m "feat(export): add user export task schema and migration" ``` --- ### Task 2: Request Validation + Masking Rules **Files:** - Create: `server/constants/export.ts` - Create: `server/utils/me-export-request-body.ts` - Create: `server/utils/me-export-request-body.test.ts` - Create: `server/utils/export-mask.ts` - Test: `server/utils/me-export-request-body.test.ts` - [ ] **Step 1: Write failing tests for request body and masking behavior** ```ts // File: server/utils/me-export-request-body.test.ts import { describe, expect, test } from "bun:test"; import { parseMeExportRequestBody } from "./me-export-request-body"; import { sanitizeUserForExport } from "./export-mask"; describe("parseMeExportRequestBody", () => { test("defaults to masked when absent", () => { expect(parseMeExportRequestBody({})).toEqual({ maskPolicy: "masked" }); }); test("rejects invalid policy", () => { expect(() => parseMeExportRequestBody({ maskPolicy: "abc" })).toThrow(); }); }); describe("sanitizeUserForExport", () => { test("never exports password/session-like fields", () => { const sanitized = sanitizeUserForExport({ username: "u", password: "hash", sessionToken: "x" } as any, "raw"); expect("password" in sanitized).toBe(false); expect("sessionToken" in sanitized).toBe(false); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `bun test server/utils/me-export-request-body.test.ts` Expected: FAIL with missing modules/functions. - [ ] **Step 3: Implement parser + masking utility** ```ts // File: server/utils/me-export-request-body.ts export type ExportMaskPolicy = "masked" | "raw"; export function parseMeExportRequestBody(body: unknown): { maskPolicy: ExportMaskPolicy } { const payload = (body && typeof body === "object" ? body : {}) as { maskPolicy?: unknown }; const raw = payload.maskPolicy; if (raw === undefined || raw === null || raw === "") return { maskPolicy: "masked" }; if (raw === "masked" || raw === "raw") return { maskPolicy: raw }; throw createError({ statusCode: 400, statusMessage: "maskPolicy 仅支持 masked/raw" }); } ``` ```ts // File: server/utils/export-mask.ts const NEVER_EXPORT_KEYS = new Set([ "password", "passwordHash", "sessionToken", "refreshToken", "resetToken", "apiKey", "secretKey", ]); export function sanitizeUserForExport>(input: T, policy: "masked" | "raw") { const out: Record = {}; for (const [k, v] of Object.entries(input)) { if (NEVER_EXPORT_KEYS.has(k)) continue; out[k] = policy === "masked" ? maskField(k, v) : v; } return out; } function maskField(key: string, value: unknown): unknown { if (typeof value !== "string") return value; if (key.includes("email")) return value.replace(/^(.).+(@.+)$/, "$1***$2"); if (key.includes("phone")) return value.replace(/^(\d{3})\d+(\d{2})$/, "$1****$2"); if (key.toLowerCase().includes("ip")) return value.replace(/^(\d+\.\d+)\.\d+\.\d+$/, "$1.*.*"); return value; } ``` - [ ] **Step 4: Run tests to verify pass** Run: `bun test server/utils/me-export-request-body.test.ts` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add server/constants/export.ts \ server/utils/me-export-request-body.ts \ server/utils/me-export-request-body.test.ts \ server/utils/export-mask.ts git commit -m "feat(export): add request validation and masking utilities" ``` --- ### Task 3: Export Job Service (create/list/update) **Files:** - Create: `server/service/export/jobs.ts` - Modify: `server/service/export/jobs.test.ts` - Test: `server/service/export/jobs.test.ts` - [ ] **Step 1: Write failing tests for job lifecycle** ```ts // File: server/service/export/jobs.test.ts import { describe, expect, test } from "bun:test"; import { createExportTask, listExportTasksByUser, markExportTaskRunning } from "./jobs"; describe("export jobs service", () => { test("createExportTask creates queued task", async () => { const task = await createExportTask({ userId: 1, maskPolicy: "masked" }); expect(task.status).toBe("queued"); expect(task.maskPolicy).toBe("masked"); }); test("markExportTaskRunning sets cutoff timestamp", async () => { const task = await createExportTask({ userId: 1, maskPolicy: "raw" }); const updated = await markExportTaskRunning(task.id); expect(updated.status).toBe("running"); expect(updated.exportCutoffAt).toBeTruthy(); }); test("listExportTasksByUser only returns own tasks", async () => { const rows = await listExportTasksByUser(1); expect(Array.isArray(rows)).toBe(true); }); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `bun test server/service/export/jobs.test.ts` Expected: FAIL with missing implementation. - [ ] **Step 3: Implement minimal job service** ```ts // File: server/service/export/jobs.ts import { dbGlobal } from "drizzle-pkg/lib/db"; import { userExportTasks } from "drizzle-pkg/lib/schema/export"; import { and, desc, eq } from "drizzle-orm"; import { nextIntegerId } from "#server/utils/sqlite-id"; export async function createExportTask(params: { userId: number; maskPolicy: "masked" | "raw" }) { const id = await nextIntegerId(userExportTasks, userExportTasks.id); await dbGlobal.insert(userExportTasks).values({ id, userId: params.userId, maskPolicy: params.maskPolicy, status: "queued", }); const [row] = await dbGlobal.select().from(userExportTasks).where(eq(userExportTasks.id, id)).limit(1); return row!; } export async function listExportTasksByUser(userId: number) { return dbGlobal.select().from(userExportTasks).where(eq(userExportTasks.userId, userId)).orderBy(desc(userExportTasks.id)); } export async function markExportTaskRunning(taskId: number) { const cutoff = new Date(); await dbGlobal.update(userExportTasks).set({ status: "running", exportCutoffAt: cutoff }).where(eq(userExportTasks.id, taskId)); const [row] = await dbGlobal.select().from(userExportTasks).where(eq(userExportTasks.id, taskId)).limit(1); return row!; } export async function markExportTaskSucceeded(taskId: number, payload: { outputDir: string; outputName: string; totalBytes: number; expiresAt: Date }) { await dbGlobal .update(userExportTasks) .set({ status: "succeeded", ...payload }) .where(eq(userExportTasks.id, taskId)); } export async function markExportTaskFailed(taskId: number, payload: { errorCode: string; errorMessage: string }) { await dbGlobal.update(userExportTasks).set({ status: "failed", ...payload }).where(eq(userExportTasks.id, taskId)); } export async function getExportTaskForUser(taskId: number, userId: number) { const [row] = await dbGlobal .select() .from(userExportTasks) .where(and(eq(userExportTasks.id, taskId), eq(userExportTasks.userId, userId))) .limit(1); return row ?? null; } ``` - [ ] **Step 4: Run tests to verify pass** Run: `bun test server/service/export/jobs.test.ts` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add server/service/export/jobs.ts server/service/export/jobs.test.ts git commit -m "feat(export): add export task lifecycle service" ``` --- ### Task 4: Build Export Artifacts (data/files/manifest) **Files:** - Create: `server/service/export/build-data.ts` - Create: `server/service/export/build-files.ts` - Create: `server/service/export/build-manifest.ts` - Create: `server/service/export/build-manifest.test.ts` - Create: `server/utils/export-hash.ts` - Test: `server/service/export/build-manifest.test.ts` - [ ] **Step 1: Write failing test for manifest shape + checksum keys** ```ts // File: server/service/export/build-manifest.test.ts import { describe, expect, test } from "bun:test"; import { buildExportManifest } from "./build-manifest"; describe("buildExportManifest", () => { test("outputs schemaVersion and checksums", () => { const manifest = buildExportManifest({ schemaVersion: 1, userId: 1, maskPolicy: "masked", exportCutoffAt: new Date("2026-04-24T00:00:00.000Z"), dataChecksums: [{ file: "data/user.json", sha256: "a" }], fileChecksums: [{ file: "files/a.webp", sha256: "b" }], stats: { dataRows: 10, files: 1, bytes: 1234 }, }); expect(manifest.schemaVersion).toBe(1); expect(manifest.checksums.files[0].file).toBe("files/a.webp"); }); }); ``` - [ ] **Step 2: Run test to verify fail** Run: `bun test server/service/export/build-manifest.test.ts` Expected: FAIL with missing module/function. - [ ] **Step 3: Implement data/files/manifest builders** ```ts // File: server/service/export/build-data.ts (core signature) export async function buildExportDataJson(params: { userId: number; cutoffAt: Date; maskPolicy: "masked" | "raw"; outputDataDir: string; }): Promise<{ files: Array<{ file: string; rowCount: number }> }> { // 1) query users/profile/posts/timeline/comments/mediaAssets/mediaRefs up to cutoffAt // 2) sanitize with export-mask policy // 3) write JSON files into outputDataDir // 4) return file list + row counts } ``` ```ts // File: server/service/export/build-files.ts (core signature) export async function buildExportMediaFiles(params: { userId: number; cutoffAt: Date; outputFilesDir: string; }): Promise<{ files: Array<{ file: string; bytes: number }> }> { // query media_assets by user, copy each storageKey into files/, keep deterministic relative paths } ``` ```ts // File: server/service/export/build-manifest.ts export function buildExportManifest(input: { schemaVersion: number; userId: number; maskPolicy: "masked" | "raw"; exportCutoffAt: Date; dataChecksums: Array<{ file: string; sha256: string }>; fileChecksums: Array<{ file: string; sha256: string }>; stats: { dataRows: number; files: number; bytes: number }; }) { return { schemaVersion: input.schemaVersion, exportedAt: new Date().toISOString(), exportCutoffAt: input.exportCutoffAt.toISOString(), userId: input.userId, maskPolicy: input.maskPolicy, stats: input.stats, checksums: { data: input.dataChecksums, files: input.fileChecksums }, }; } ``` - [ ] **Step 4: Run tests to verify pass** Run: `bun test server/service/export/build-manifest.test.ts` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add server/service/export/build-data.ts \ server/service/export/build-files.ts \ server/service/export/build-manifest.ts \ server/service/export/build-manifest.test.ts \ server/utils/export-hash.ts git commit -m "feat(export): add export artifact builders and manifest" ``` --- ### Task 5: Runner + Task Processor + Me APIs **Files:** - Create: `server/service/export/run.ts` - Create: `server/tasks/export/process.ts` - Create: `server/api/me/export/request.post.ts` - Create: `server/api/me/export/tasks.get.ts` - Create: `server/api/me/export/tasks/[id]/download.get.ts` - Modify: `server/service/export/jobs.ts`(补 claimNextQueuedTask) - Test: `server/service/export/jobs.test.ts` + manual API smoke - [ ] **Step 1: Write failing tests for queue claim behavior** ```ts // File: server/service/export/jobs.test.ts (append) import { claimNextQueuedTask } from "./jobs"; test("claimNextQueuedTask returns null when no queued task", async () => { const task = await claimNextQueuedTask(); expect(task).toBeNull(); }); ``` - [ ] **Step 2: Run test to verify fail** Run: `bun test server/service/export/jobs.test.ts` Expected: FAIL with missing function. - [ ] **Step 3: Implement runner and APIs** ```ts // File: server/service/export/run.ts export async function runExportTask(taskId: number): Promise { const running = await markExportTaskRunning(taskId); try { // create output dirs // build data json + files // hash files + build manifest + write manifest.json // mark succeeded with expiresAt } catch (error) { await markExportTaskFailed(taskId, { errorCode: "EXPORT_BUILD_FAILED", errorMessage: error instanceof Error ? error.message : "unknown export error", }); } } ``` ```ts // File: server/api/me/export/request.post.ts export default defineWrappedResponseHandler(async (event) => { const user = await event.context.auth.requireUser(); const body = await readBody(event); const { maskPolicy } = parseMeExportRequestBody(body); const task = await createExportTask({ userId: user.id, maskPolicy }); return R.success({ taskId: task.id, status: task.status }); }); ``` ```ts // File: server/api/me/export/tasks.get.ts export default defineWrappedResponseHandler(async (event) => { const user = await event.context.auth.requireUser(); const tasks = await listExportTasksByUser(user.id); return R.success({ items: tasks.map((t) => ({ id: t.id, status: t.status, maskPolicy: t.maskPolicy, createdAt: t.createdAt.toISOString(), expiresAt: t.expiresAt?.toISOString() ?? null, })), }); }); ``` ```ts // File: server/api/me/export/tasks/[id]/download.get.ts export default defineWrappedResponseHandler(async (event) => { const user = await event.context.auth.requireUser(); const id = Number(getRouterParam(event, "id")); const task = await getExportTaskForUser(id, user.id); if (!task || task.status !== "succeeded" || !task.outputDir) { throw createError({ statusCode: 404, statusMessage: "导出文件不存在或未完成" }); } // serve file/dir stream according to deployment convention }); ``` - [ ] **Step 4: Run tests and smoke endpoints** Run: - `bun test server/service/export/jobs.test.ts` - `bun run dev` then call: - `POST /api/me/export/request` - `GET /api/me/export/tasks` Expected: - tests PASS - request returns queued task id - list endpoint contains new task. - [ ] **Step 5: Commit** ```bash git add server/service/export/run.ts \ server/tasks/export/process.ts \ server/api/me/export/request.post.ts \ server/api/me/export/tasks.get.ts \ server/api/me/export/tasks/[id]/download.get.ts \ server/service/export/jobs.ts \ server/service/export/jobs.test.ts git commit -m "feat(export): add export runner, processor task, and me export APIs" ``` --- ### Task 6: Frontend Export Center (`/me/export`) **Files:** - Modify: `app/pages/me/index.vue` - Create: `app/pages/me/export/index.vue` - Create: `app/types/export.ts` - Test: manual browser verification - [ ] **Step 1: Write lightweight UI contract notes as failing checklist** ```md // File: app/pages/me/export/index.vue (top TODO block during implementation) - must show "发起导出" button - must allow selecting mask policy (masked/raw) - must show task list status + created time - must show download button only when succeeded ``` - [ ] **Step 2: Run app and verify page not present yet (expected fail)** Run: `bun run dev` and open `/me/export` Expected: 404 or missing route behavior. - [ ] **Step 3: Implement page and dashboard entry** ```vue ``` ```vue
数据导出

导出用户文件与业务数据用于迁移。

进入
``` - [ ] **Step 4: Manual verify pass** Run: `bun run dev` Expected: - `/me/export` 可访问 - 可创建导出任务 - 成功任务显示下载入口。 - [ ] **Step 5: Commit** ```bash git add app/pages/me/index.vue app/pages/me/export/index.vue app/types/export.ts git commit -m "feat(export): add me export center page and task actions" ``` --- ### Task 7: Security/Regression Tests + Docs **Files:** - Modify: `server/service/export/jobs.test.ts` - Modify: `server/utils/me-export-request-body.test.ts` - Modify: `README.md` - Modify: `docs/superpowers/specs/2026-04-24-user-data-backup-design.md`(实现状态备注) - Test: targeted bun tests + lint - [ ] **Step 1: Add failing security regression tests** ```ts // File: server/utils/me-export-request-body.test.ts (append) test("raw policy still excludes never-export fields", () => { const data = sanitizeUserForExport({ password: "x", apiKey: "k", email: "a@b.com" } as any, "raw"); expect(data).toEqual({ email: "a@b.com" }); }); ``` - [ ] **Step 2: Run tests to verify failure (if behavior regresses)** Run: `bun test server/utils/me-export-request-body.test.ts` Expected: PASS only when blacklist behavior is correct. - [ ] **Step 3: Update docs with usage commands** ```md ## 用户数据导出(迁移) - 发起导出:`POST /api/me/export/request`,body: `{ "maskPolicy": "masked" | "raw" }` - 查询任务:`GET /api/me/export/tasks` - 下载结果:`GET /api/me/export/tasks/:id/download` ``` - [ ] **Step 4: Final verification** Run: - `bun test server/service/export/jobs.test.ts` - `bun test server/service/export/build-manifest.test.ts` - `bun test server/utils/me-export-request-body.test.ts` Expected: all PASS. - [ ] **Step 5: Commit** ```bash git add server/service/export/jobs.test.ts \ server/utils/me-export-request-body.test.ts \ README.md \ docs/superpowers/specs/2026-04-24-user-data-backup-design.md git commit -m "test/docs(export): add security regression coverage and usage docs" ``` --- ## Self-Review Checklist - Spec coverage: - 异步任务模型:Task 1, 3, 5 - 目录+清单导出:Task 4 - 全量用户域数据 + 文件:Task 4 - 脱敏与永不导出:Task 2, 7 - 安全下载与过期:Task 5 - 前端入口与操作:Task 6 - Placeholder scan: 已避免 `TODO/TBD`;每个任务给出代码/命令/预期。 - Type consistency: - `maskPolicy` 统一 `masked | raw` - 状态统一 `queued|running|succeeded|failed|expired` - 清单版本固定 `schemaVersion: 1`