diff --git a/docs/MILESTONE_11_1_ACCOUNT_FOUNDATION_AUDIT.md b/docs/MILESTONE_11_1_ACCOUNT_FOUNDATION_AUDIT.md new file mode 100644 index 0000000..16dc6fd --- /dev/null +++ b/docs/MILESTONE_11_1_ACCOUNT_FOUNDATION_AUDIT.md @@ -0,0 +1,656 @@ +# Milestone 11.1 Account Foundation Audit + +## Scope + +This document audits the current ownership and device model in Velody and proposes a migration foundation for real user accounts without changing current behavior. + +It is based on the current Prisma schema, backend ownership/auth/upload/sync routes, and the Apple client persistence and sync code. + +Primary sources inspected: + +- `backend/prisma/schema.prisma` +- `backend/prisma/migrations/*` +- `backend/src/modules/users/*` +- `backend/src/modules/auth/*` +- `backend/src/modules/devices/*` +- `backend/src/modules/library/*` +- `backend/src/modules/sync/*` +- `backend/src/modules/uploads/*` +- `backend/src/modules/assets/*` +- `backend/src/modules/artwork/*` +- `packages/apple/VelodyNetworking/*` +- `packages/apple/VelodyPersistence/*` +- `packages/apple/VelodySync/*` +- `apps/apple/VelodyiPhone/Sources/iPhoneLibraryViewModel.swift` +- `apps/apple/VelodyMac/Sources/MacLibraryViewModel.swift` + +## 1. Current Ownership Model Audit + +### Executive summary + +- The backend already has a real ownership boundary, but it is named `User` and currently behaves as a library owner container, not a full end-user account. +- All important remote library entities are already scoped by `userId`. +- Protected content routes already trust device bearer tokens, not raw `deviceId` query/body values. +- The biggest missing pieces for real accounts are identity tables, safe device linking, account-scoped client caches, and a safe ownership transfer story. +- The biggest near-term risk is that legacy owner fallback still exists in `OwnerContext`, and `devices/register` still uses that fallback-enabled path. + +### Current root owner + +`User` is the current root ownership record. + +Current fields: + +- `id` +- `slug` +- `displayName` +- `isDefault` +- `libraryCursor` +- timestamps + +Current meaning: + +- `default-owner` is auto-created by `DefaultUserService`. +- There is no email, password hash, OAuth identity, verification state, or recovery state. +- In practice, `User` means "library owner" today, not "signed-in human account". + +### Current relationship graph + +```mermaid +graph TD + OC["OwnerContext"] --> U["User"] + D["Device"] --> U + T["Track"] --> U + AA["AudioAsset"] --> U + AW["ArtworkAsset"] --> U + US["UploadSession"] --> U + LE["LibraryEvent"] --> U + DSC["DeviceSyncCursor"] --> U + + T -->|"primaryAudioAssetId"| AA + T -->|"artworkAssetId"| AW + AA -->|"trackId"| T + AW -->|"shared by many tracks"| T + US --> D + AA -->|"sourceDeviceId"| D + DSC --> D +``` + +### Current entity ownership and behavior + +| Entity | Current owner link | Notes | +| --- | --- | --- | +| `User` | Root owner | One special legacy row: `default-owner`. | +| `Device` | `device.userId -> users.id` | Device bearer token is the active auth credential for protected routes. | +| `Track` | `track.userId -> users.id` | Logical library item. Has one optional primary audio asset and one optional artwork asset. | +| `AudioAsset` | `audio_asset.userId -> users.id` | Deduped by `(userId, sha256)`. Same bytes across different users are treated as different owned assets. | +| `ArtworkAsset` | `artwork_asset.userId -> users.id` | Deduped by `(userId, sha256)`. One artwork asset can back multiple tracks. | +| `UploadSession` | `upload_session.userId -> users.id` and `deviceId` | Uploads are already owner-scoped and device-attributed. | +| `LibraryEvent` | `library_event.userId -> users.id` | Incremental sync feed is per owner. | +| `DeviceSyncCursor` | `device_sync_cursor.userId -> users.id` and `deviceId` | Server remembers per-device cursor progress inside an owner scope. | + +### How ownership is resolved today + +Current resolution order inside `BootstrapOwnerContextService`: + +1. Authenticated device from request context. +2. Legacy raw `deviceId` captured from request body/query. +3. Bootstrap default owner. + +Important details: + +- `RequestContextMiddleware` captures raw `deviceId` before DTO validation. +- `ProtectedDeviceAuthMiddleware` requires `Authorization: Bearer ` on protected routes. +- `DeviceAuthService` resolves bearer token to `Device.id + Device.userId`. +- Protected routes then use the authenticated device's `userId`. +- `deviceId` fields still exist in several DTOs, but they are now metadata only for most protected routes. + +### Where legacy fallback is still active + +Protected content routes do not rely on legacy fallback anymore, but `devices/register` still calls: + +- `ownerContext.resolve()` with default options + +That means registration currently behaves like this: + +- If a bearer token is present, the new device joins that owner's `userId`. +- If no bearer token is present, fallback can still consult raw `deviceId` from the request context before using `default-owner`. + +This is the main ownership-resolution gap that should be isolated before real accounts are exposed. + +### Route-by-route ownership model + +Protected and device-token scoped today: + +- `/api/v1/library/*` +- `/api/v1/sync/*` +- `/api/v1/uploads/*` +- `/api/v1/assets/*` +- `/api/v1/artwork/*` +- `/api/v1/devices/heartbeat` + +Behavior: + +- Authorization comes from device bearer token. +- Returned or mutated content is filtered by the authenticated device's `userId`. +- Cross-owner access is rejected by user checks in services and tests. + +Special case: + +- `/api/v1/devices/register` is public with optional auth. +- This is the only route that still uses the broader owner fallback chain. + +### Upload flow and ownership + +The upload pipeline is already strongly owner-scoped. + +Current behavior: + +- `prepare` looks up `AudioAsset` by `(userId, sha256)`. +- Duplicate bytes are reused only inside the same owner. +- New binary files are stored under `users//audio/.mp3`. +- Artwork is stored under `users//artwork/.`. +- `finalize` creates or updates: + - `Track` + - `AudioAsset` + - `ArtworkAsset` + - `LibraryEvent` +- `AudioAsset.sourceDeviceId` records which device uploaded the bytes. + +Important current invariant: + +- Within one owner, identical audio SHA currently collapses to one `AudioAsset`. +- In practice, the finalize path also tends to collapse duplicate uploads onto the same track/library item for that owner. + +### Incremental sync and ownership + +The sync model is already per-owner. + +Current behavior: + +- `User.libraryCursor` is the per-owner cursor counter. +- `LibraryEvent.cursor` is unique per user. +- `sync/bootstrap` returns the full current remote library for the authenticated device's owner. +- `sync/changes` returns only events for that owner after the requested cursor. +- If the cursor is too old, the API returns `requiresBootstrap: true`. +- Server-side `DeviceSyncCursor` is updated after bootstrap and after incremental changes. + +Important current invariant: + +- The sync model assumes the device remains inside one stable owner namespace while its cursor is valid. + +### Favorites, offline downloads, artwork persistence, and local sync state + +These are not server-owned today. They are client-local only. + +On iPhone: + +- Remote library cache is stored in `remote-library.json`. +- Sync cursor is stored in `remote-library-sync-cursor.json`. +- Download state is stored in `remote-download-states.json`. +- Favorites are stored in `favorite-tracks.json`. +- Offline audio files are stored under `Application Support/Velody/audio/.mp3`. +- Cached artwork is stored under `Application Support/Velody/artwork/.`. + +Current client keys: + +- Favorites are keyed by `remoteTrackId`. +- Download state is keyed by `remoteTrackId` and tracks the current `assetId`. +- Artwork cache is keyed by `artworkId`. +- None of these stores are namespaced by owner or account. + +On Mac: + +- The local scan catalog persists `uploadStatus` and `remoteTrackId`. +- That persistence also has no explicit owner/account namespace. + +Implication: + +- Current local persistence assumes one active owner per app install. +- Any future account switch on the same install would mix cached library data, favorites, sync cursors, downloads, artwork, and uploaded-track metadata unless namespacing is added first. + +### Current assets that look account-ready already + +- `User` as a root owner row. +- `Device.userId` many-to-one relationship. +- User-scoped content tables and user-scoped storage layout. +- Device bearer token auth. +- Per-device sync cursor tracking. +- Library event log with bootstrap fallback. + +### Current gaps + +- No real identity model. +- No account lifecycle model. +- No account recovery model. +- No safe device-link or device-relink API. +- No account-scoped client cache namespace. +- No explicit transfer model for moving or claiming ownership. +- `bootstrapToken` is returned and stored, but not used after registration. + +## 2. What Can Be Reused For Real User Accounts + +### Backend pieces that are already reusable + +- `User` can become the logical account owner container with minimal schema churn. +- `Device` already models multiple devices per owner. +- Protected library/upload/download/artwork routes are already device-token based and owner-filtered. +- `Track`, `AudioAsset`, `ArtworkAsset`, `UploadSession`, `LibraryEvent`, and `DeviceSyncCursor` are already account-shaped because they hang off `userId`. +- Per-owner storage paths already prevent cross-owner file leakage. +- Incremental sync already has a stable per-owner event log and bootstrap reset path. + +### Good migration property in the current schema + +The existing library data model does not need to be rewritten from "single global library" to "account library". That rewrite already happened when `userId` was added across the schema. + +### Existing auth pieces worth reusing + +- Device access tokens should remain the credential for content APIs. +- `tokenLastUsedAt` and `tokenRevokedAt` are already useful for device management. +- The hashed `installTokenHash` can become the continuity proof for device recovery or guest-to-account claim flows. + +### Existing client pieces worth reusing + +- The iPhone sync/download/offline model can stay structurally the same. +- The client already tolerates bootstrap fallback and replay-safe incremental sync. +- Artwork and offline file resilience logic is already strong enough for multi-device account behavior after namespacing is added. + +## 3. Migration Strategy + +### Guiding principles + +1. Keep current legacy devices working until they explicitly migrate. +2. Do not replace device-token auth for library APIs with user-session auth. +3. Add real account auth beside device auth, then link devices into accounts. +4. Namespace local caches before enabling account switching. +5. Treat ownership transfer as an explicit audited operation, not a side effect. + +### Guest/default owner compatibility + +Recommended approach: + +- Keep the current `default-owner` row as a legacy compatibility owner. +- Introduce an explicit account kind so the system can distinguish: + - legacy default owner + - guest owner + - registered account +- For new anonymous installs in the post-account world, prefer per-install guest owners over the shared `default-owner`. +- Keep `default-owner` only for existing legacy devices and migrations. + +Why: + +- Shared bootstrap ownership is acceptable for a private single-owner system. +- It is not a safe long-term public guest model. + +### Email/password later + +Recommended approach: + +- Add password/email tables without changing content ownership tables. +- Authenticate the human account first. +- Then link the current device to that account and rotate the device access token. +- Keep uploads, library sync, asset download, and artwork download on device bearer auth. + +### OAuth later + +Recommended approach: + +- Add external identity rows per provider. +- Reuse the same post-auth device linking flow used by email/password. +- Do not build separate content authorization rules for OAuth devices. + +### Multiple devices per account + +This is already structurally supported. + +Needed additions: + +- device listing +- device revoke/rotate endpoints +- link-current-device flow +- trusted-device or link-code flow later if desired + +### Recovery + +Account recovery: + +- Email/password accounts should get standard password-reset and email-verification flows. +- OAuth accounts recover through the provider plus a device re-link flow. + +Device recovery: + +- Reuse `bootstrapToken` and `installTokenHash` as an install continuity secret. +- Use it to reissue a device access token after successful account auth or trusted-device approval. +- Rotate device tokens when a device is re-linked or recovered. + +### Safe ownership transfer + +Recommended first-release strategy for legacy default-owner migration: + +- Prefer copy-on-claim over destructive move. + +Reason: + +- The current `default-owner` can still have multiple legacy devices. +- Copy-on-claim avoids immediately breaking old devices and gives rollback room. + +Recommended later strategy for explicit account merges or destructive moves: + +- Introduce an `OwnershipTransfer` job with audit data. +- Lock source and target owners during transfer. +- Dedupe target-side `AudioAsset` and `ArtworkAsset` rows by the target owner's unique constraints. +- Rebuild or reset sync state after transfer. +- Rotate device tokens for moved devices. + +## 4. Target Architecture + +### Target data model + +Recommended interpretation: + +- Keep the physical `users` table for now. +- Treat it as the account-owner table in the application layer. +- If desired later, rename the Prisma model to `Account` while keeping `@@map("users")` to avoid a risky table rename. + +Recommended entities: + +#### Keep and extend `User` + +Add fields such as: + +- `accountKind` enum + - `LEGACY_DEFAULT` + - `GUEST` + - `REGISTERED` +- `accountStatus` enum + - `ACTIVE` + - `LOCKED` + - `DELETED` +- `libraryNamespace` UUID + - rotated when ownership scope changes in a way that invalidates client caches or cursors +- `claimedAt` nullable timestamp + +Keep existing owner relations: + +- `devices` +- `tracks` +- `audioAssets` +- `artworkAssets` +- `uploadSessions` +- `libraryEvents` +- `syncCursors` + +#### Keep `Device` as the install credential + +Keep: + +- `userId` +- bearer token fields +- install token hash +- heartbeat metadata + +Add: + +- `linkedAt` +- `relinkedAt` +- optional device status if needed + +Recommendation: + +- Keep `Device.userId` as the current owner link for compatibility. +- Add separate history/audit rows for link changes instead of relying only on in-place mutation. + +#### Add identity tables + +Recommended new tables: + +- `UserPasswordCredential` + - `userId` + - `emailNormalized` + - `passwordHash` + - `emailVerifiedAt` + - timestamps +- `UserOAuthIdentity` + - `userId` + - `provider` + - `providerSubject` + - `emailAtProvider` + - timestamps +- `EmailVerificationToken` +- `PasswordResetToken` + +#### Add transfer and link audit tables + +Recommended new tables: + +- `DeviceLinkHistory` + - `deviceId` + - `userId` + - `linkMethod` + - `linkedAt` + - `unlinkedAt` +- `OwnershipTransfer` + - `sourceUserId` + - `targetUserId` + - `mode` (`COPY`, `MOVE`, `MERGE`) + - `status` + - `initiatedByUserId` + - `initiatedByDeviceId` + - summary payload + - timestamps + +### Recommended Prisma changes + +Minimal-content-table change strategy: + +- Do not change the ownership foreign keys on `Track`, `AudioAsset`, `ArtworkAsset`, `UploadSession`, `LibraryEvent`, or `DeviceSyncCursor`. +- Add identity and audit tables around them. +- Add `User.accountKind`, `User.accountStatus`, and `User.libraryNamespace`. +- Keep `User.isDefault` during transition for compatibility, but treat it as legacy-only. + +Important recommended semantic change: + +- `OwnerContext` should evolve into an account context that does not use raw `deviceId` fallback except on explicit legacy migration endpoints. + +### Recommended API changes + +#### Keep existing content APIs device-token based + +Keep: + +- `/devices/heartbeat` +- `/library/*` +- `/sync/*` +- `/uploads/*` +- `/assets/*` +- `/artwork/*` + +Recommendation: + +- These APIs should continue to authorize through device bearer tokens. +- Real account auth should not replace this layer. + +#### Add account/session APIs + +Recommended: + +- `POST /api/v1/auth/signup/password` +- `POST /api/v1/auth/login/password` +- `POST /api/v1/auth/logout` +- `POST /api/v1/auth/password-reset/start` +- `POST /api/v1/auth/password-reset/complete` +- OAuth start/callback endpoints later + +#### Add account context API + +Recommended: + +- `GET /api/v1/me` + +Suggested response: + +- `accountId` +- `accountKind` +- `displayName` +- `libraryNamespace` +- `currentDeviceId` + +Why: + +- Clients need a stable owner namespace before caching account-specific state. + +#### Add device linking APIs + +Recommended: + +- `POST /api/v1/devices/link-current` +- `GET /api/v1/devices` +- `POST /api/v1/devices/:deviceId/revoke` +- `POST /api/v1/devices/recover-token` + +#### Add legacy-owner claim/transfer APIs + +Recommended: + +- `POST /api/v1/ownership/claim-legacy-default` +- `POST /api/v1/ownership/transfers` +- `GET /api/v1/ownership/transfers/:transferId` + +### Target auth flow + +Recommended steady-state flow: + +1. Device registers and gets a device access token plus bootstrap token. +2. User authenticates with password or OAuth using account/session endpoints. +3. Client calls `link-current-device`. +4. Server links or upgrades the current device into the authenticated account. +5. Server rotates the device access token and returns account context, including `libraryNamespace`. +6. Client switches its local storage namespace and runs sync bootstrap. + +Why this is the least disruptive path: + +- Existing content APIs stay the same. +- Device auth keeps working for sync/download/upload. +- Account auth is only responsible for account lifecycle and device linking. + +### Target device linking flow + +Recommended first implementation: + +1. Anonymous or legacy device already exists. +2. User completes account auth. +3. Client sends: + - current device bearer token + - account session auth + - optional transfer preference (`copy_legacy_library`, `move_legacy_library`, `link_only`) +4. Server: + - verifies both contexts + - decides whether to keep guest/default owner content, copy it, or move it + - relinks the device + - rotates device token + - bumps `libraryNamespace` if the owner scope changed +5. Client: + - stores new token + - clears or re-namespaces owner-scoped caches + - boots the synced library again + +### Required client-side storage changes + +Before account switching is supported, move these stores under an owner/account namespace: + +- remote library cache +- remote library sync cursor +- remote download state +- favorites +- offline audio files +- cached artwork +- Mac upload status and `remoteTrackId` persistence + +Suggested namespace root: + +- `accounts//...` + +This namespace should be derived from the server, not guessed locally. + +## 5. Risks And Recommended Implementation Order For Milestone 11.2+ + +### Key risks + +1. `devices/register` still uses fallback-enabled owner resolution. + This is the main current ownership-resolution risk and should be narrowed before public account flows. + +2. Client caches are global, not owner-scoped. + Without namespacing, account switching on the same install will mix remote library data, cursors, favorites, downloads, artwork, and upload metadata. + +3. Ownership transfer can break incremental sync. + Current cursors assume a stable owner scope. Re-linking devices or rebuilding a library needs a namespace or epoch reset. + +4. User-scoped storage keys make transfer non-trivial. + Moving assets between owners means updating DB ownership and also copying or moving files plus `storageKey` values. + +5. Transfer must respect target-side dedup rules. + `AudioAsset` and `ArtworkAsset` are unique per `(userId, sha256)`, so transfer logic needs deterministic merge behavior. + +6. `bootstrapToken` exists but is currently inert. + The project should either formalize it as a recovery/install secret or remove it from the conceptual model. + +7. Mac local upload state is not account-aware. + Persisted `remoteTrackId` and upload status will become misleading after account changes unless they are namespaced or reset. + +### Recommended implementation order + +#### Milestone 11.2 + +1. Add account foundation fields and tables. + - `User.accountKind` + - `User.accountStatus` + - `User.libraryNamespace` + - password identity tables + - OAuth identity tables + - recovery token tables + - device link history + - ownership transfer audit table + +2. Add `GET /api/v1/me`. + Clients need account identity and namespace before any account-aware caching can be correct. + +3. Remove or isolate legacy fallback from public registration. + Keep raw `deviceId` fallback only on explicit legacy claim endpoints if it is still needed at all. + +4. Make client persistence account-scoped before shipping account switching. + Do this on iPhone and Mac before login/relink UX becomes user-visible. + +#### Milestone 11.3 + +1. Add password account creation and login APIs. +2. Add `link-current-device`. +3. Rotate device tokens during link/relink. +4. Force bootstrap when `libraryNamespace` changes. + +#### Milestone 11.4 + +1. Ship legacy default-owner claim flow. +2. Prefer copy-on-claim first. +3. Add transfer audit visibility and rollback-safe cleanup. + +#### Milestone 11.5 + +1. Add OAuth providers. +2. Reuse the same device linking and namespace reset flow. + +#### Milestone 11.6 + +1. Add device management and recovery. +2. Add revoke, recover-token, and trusted-device link options. + +## Bottom line + +Velody is already closer to account-ready than the current UX suggests because the backend content model is already owner-scoped by `userId`. + +The most important foundation work is not rewriting library entities. It is: + +- formalizing identity around the existing owner model +- removing broad legacy owner fallback from public flows +- introducing an explicit account namespace for client persistence and sync invalidation +- making ownership transfer an explicit audited workflow + +If those pieces land first, email/password, OAuth, multiple devices per account, recovery, and safe migration away from `default-owner` can be layered on without breaking the current library, upload, or offline behavior.