Skip to main content

Deterministic Sync Architecture

Philosophy​

Velocity is the Single Source of Truth

Velocity maintains authoritative data about projects, people, and their relationships. Jira and Bitbucket are treated as external data sources that are linked to Velocity entities through manually declared relationships.

External systems (Jira, Bitbucket) are often messy:

  • Different usernames across systems (e.g., "john.doe" in Jira, "jdoe" in Bitbucket)
  • Duplicated accounts (same person, multiple accounts)
  • Inconsistent project naming (project keys change, projects merge/split)
  • Orphaned data (deleted projects, inactive accounts)

By making Velocity the source of truth and requiring manual link declarations, we ensure:

  • ✅ Data integrity: No ambiguous relationships
  • ✅ Explicit ownership: Clear mapping between Velocity and external systems
  • ✅ Flexibility: Handle edge cases and inconsistencies
  • ✅ Auditability: Know exactly what links to what

Relationship Model​

1:N Relationships​

Most relationships are one-to-many (1:N):

  • 1 Project → N Jira Projects: A Velocity project can link to multiple Jira projects
  • 1 Project → N Bitbucket Repositories: A Velocity project can link to multiple repositories
  • 1 Person → N Jira Account IDs: A person might have multiple Jira accounts (legacy, different orgs)
  • 1 Person → N Bitbucket Usernames: A person might have multiple Bitbucket accounts

Links are stored directly in the entity (Project or Person) as arrays of link objects.

Current Structure​

Projects store links to external systems in the following fields:

interface Project {
  // Jira Links (1:N)
  jiraProjects: LinkedJiraProject[];
  
  // Bitbucket Links (1:N)
  repositories: LinkedRepository[];
  
  // Legacy single-link fields (for backward compatibility)
  jiraKey?: string;           // Primary Jira project key
  jiraProjectId?: string;      // Primary Jira project ID
  bitbucketWorkspace?: string; // Primary workspace
  bitbucketRepos: string[];    // Primary repos (legacy array)
}
/**
 * Linked Jira Project
 * Stores the relationship between a Velocity project and a Jira project
 */
interface LinkedJiraProject {
  // External identifiers (from Jira)
  key: string;              // Jira project key (e.g., "PROJ")
  jiraProjectId: string;    // Jira's internal project ID
  name: string;             // Project name from Jira
  url: string;              // Jira project URL
  
  // Velocity metadata
  linkedAt: Date;           // When this link was created
  linkedBy: string;         // Person ID who created the link
  isPrimary: boolean;       // Primary Jira project for this Velocity project
  active: boolean;          // Whether to sync from this Jira project
  
  // Auto-inferred metadata (updated during sync)
  openIssues?: number;      // Current open issues count
  totalIssues?: number;     // Total issues count
  lastSynced?: Date;        // Last successful sync from this Jira project
}

/**
 * Linked Repository
 * Stores the relationship between a Velocity project and a Git repository
 */
interface LinkedRepository {
  // External identifiers (from Git platform)
  name: string;             // Repository name
  slug: string;             // Repository slug/identifier
  platform: 'github' | 'gitlab' | 'bitbucket';
  workspace?: string;       // Bitbucket workspace (if applicable)
  url: string;              // Repository URL
  
  // Velocity metadata
  linkedAt: Date;           // When this link was created
  linkedBy: string;         // Person ID who created the link
  isPrimary: boolean;       // Primary repository for this Velocity project
  active: boolean;          // Whether to sync from this repository
  
  // Auto-inferred metadata (updated during sync)
  lastCommit?: Date;        // Last commit date
  defaultBranch?: string;   // Default branch name
  lastSynced?: Date;        // Last successful sync from this repository
}

Links are managed through:

  1. Manual Creation: UI allows users to search and link external entities
  2. Auto-Inference: System suggests links based on:
    • Project name similarity
    • Repository name patterns
    • Commit message analysis
    • Issue key patterns in commits
  3. Manual Confirmation: All links require explicit user confirmation
  4. Link Validation: During sync, validate that linked entities still exist

Current Structure​

Persons store links to external accounts:

interface Person {
  // External account links (1:N support)
  jiraAccountIds: string[];        // Array of Jira account IDs
  bitbucketUsernames: string[];     // Array of Bitbucket usernames
  slackUserIds: string[];           // Array of Slack user IDs
  
  // Legacy single-link fields (for backward compatibility)
  jiraAccountId?: string;           // Primary Jira account ID
  bitbucketUsername?: string;       // Primary Bitbucket username
  slackUserId?: string;             // Primary Slack user ID
}
/**
 * External Account Link
 * Stores the relationship between a Velocity person and an external account
 */
interface ExternalAccountLink {
  // External identifiers
  platform: 'jira' | 'bitbucket' | 'slack' | 'github' | 'gitlab';
  externalId: string;       // Account ID, username, or user ID from external system
  displayName?: string;     // Display name from external system
  email?: string;           // Email from external system (if available)
  
  // Velocity metadata
  linkedAt: Date;           // When this link was created
  linkedBy: string;        // Person ID who created the link
  isPrimary: boolean;       // Primary account for this platform
  active: boolean;          // Whether to sync data from this account
  
  // Auto-inferred metadata (updated during sync)
  lastSeen?: Date;          // Last activity from this account
  lastSynced?: Date;        // Last successful sync
  verified: boolean;        // Whether link has been verified (account exists)
}

Enhanced Person Structure​

interface Person {
  // ... existing fields ...
  
  // External account links (1:N)
  externalAccounts: ExternalAccountLink[];
  
  // Helper methods (computed properties)
  // getPrimaryJiraAccountId(): string | undefined
  // getPrimaryBitbucketUsername(): string | undefined
  // getAllJiraAccountIds(): string[]
  // getAllBitbucketUsernames(): string[]
}

Sync Process​

Sync Flow​

  1. Read Velocity Links: Load project/person entities and their declared links
  2. Validate Links: Check that linked external entities still exist
  3. Fetch External Data: Use links to fetch data from Jira/Bitbucket
  4. Store External IDs: Store external identifiers in synced data
  5. Update Link Metadata: Update lastSynced, verified, etc. in links

Sync Rules​

  1. Only sync from declared links: If a project doesn't have a Jira link, don't sync Jira data for it
  2. Respect active flag: Only sync from links where active === true
  3. Store external IDs: All synced data must include the external identifier
  4. Link validation: If a link becomes invalid (entity deleted), mark it as verified: false

Example: Project Sync​

// 1. Load Velocity project with links
const project = await getProject('PROJ');
// project.jiraProjects = [
//   { key: 'PROJ', jiraProjectId: '12345', active: true, ... },
//   { key: 'PROJ-LEGACY', jiraProjectId: '67890', active: false, ... }
// ]

// 2. Filter active links
const activeJiraLinks = project.jiraProjects.filter(link => link.active);

// 3. Sync from each active link
for (const link of activeJiraLinks) {
  // Fetch issues from this Jira project
  const issues = await jiraClient.getIssues(link.key);
  
  // Store with external ID reference
  for (const issue of issues) {
    await saveIssue({
      jiraIssueId: issue.id,        // External ID
      jiraProjectId: link.jiraProjectId,  // Link to Jira project
      velocityProjectId: project.id,       // Link to Velocity project
      // ... other fields
    });
  }
  
  // Update link metadata
  await updateLink(link, {
    lastSynced: new Date(),
    verified: true,
    openIssues: issues.filter(i => i.status !== 'Done').length,
    totalIssues: issues.length,
  });
}

Example: Person Sync​

// 1. Load Velocity person with links
const person = await getPerson('person-123');
// person.externalAccounts = [
//   { platform: 'jira', externalId: 'account-123', active: true, ... },
//   { platform: 'jira', externalId: 'account-456', active: false, ... }, // Legacy account
//   { platform: 'bitbucket', externalId: 'jdoe', active: true, ... }
// ]

// 2. Filter active links
const activeJiraLinks = person.externalAccounts
  .filter(link => link.platform === 'jira' && link.active);

// 3. Sync worklogs from all active Jira accounts
for (const link of activeJiraLinks) {
  const worklogs = await jiraClient.getWorklogs(link.externalId);
  
  for (const worklog of worklogs) {
    await saveWorklog({
      jiraWorklogId: worklog.id,           // External ID
      jiraAccountId: link.externalId,      // Link to Jira account
      velocityPersonId: person.id,         // Link to Velocity person
      // ... other fields
    });
  }
}

Link Discovery & Auto-Inference​

Auto-Inference Rules​

The system can suggest links but never auto-create them:

  1. Project Name Matching:

    • "Mobile App" project → suggests Jira project "MOBILE"
    • Fuzzy matching on project names

  2. Repository Pattern Matching:

    • Project "Platform" → suggests repos matching "platform-*"
    • Pattern: project-name-*, *-project-name

  3. Commit Message Analysis:

    • Commits mentioning "PROJ-123" → suggests linking repo to PROJ project
    • Extract Jira keys from commit messages

  4. Issue Key Patterns:

    • PRs/commits with issue keys → suggests linking repo to that Jira project

  5. Email Matching:

    • Person email "john@example.com" → suggests Jira account with same email
    • Bitbucket account with same email
┌─────────────────────────────────────────┐
│ Link Jira Projects to "Mobile App"      │
├─────────────────────────────────────────┤
│                                         │
│ Suggested Links:                        │
│ ☑ MOBILE - Mobile Application          │
│ ☐ MOBILE-LEGACY - Legacy Mobile        │
│ ☐ MOB - Mobile (deprecated)             │
│                                         │
│ Search for other projects:              │
│ [Search Jira projects...]              │
│                                         │
│ [Cancel]  [Confirm Links]               │
└─────────────────────────────────────────┘

Before confirming a link:

  1. Verify existence: Check that external entity exists
  2. Check conflicts: Ensure no other Velocity entity links to this external entity
  3. Show preview: Display what data will be synced
  4. Warn on duplicates: Alert if multiple external entities map to same Velocity entity

Data Storage​

Synced Data Must Include External IDs​

All synced data (issues, worklogs, commits, PRs) must store:

  1. External identifier: The ID from the external system
  2. Velocity entity reference: Link back to Velocity project/person
  3. Link reference: Which link was used to sync this data
interface Issue {
  // External identifiers (from Jira)
  jiraIssueId: string;      // Jira's internal issue ID
  issueKey: string;          // Jira issue key (e.g., "PROJ-123")
  
  // Velocity references
  velocityProjectId: string; // Link to Velocity project
  jiraProjectLinkKey: string; // Which jiraProjects[] entry was used
  
  // ... other fields
}

interface Worklog {
  // External identifiers (from Jira)
  jiraWorklogId: string;    // Jira's worklog ID
  
  // Velocity references
  velocityPersonId: string; // Link to Velocity person
  jiraAccountLinkId: string; // Which externalAccounts[] entry was used
  
  // ... other fields
}

Migration Strategy​

  1. Add jiraProjects[] and repositories[] arrays to Project type
  2. Add externalAccounts[] array to Person type
  3. Keep legacy fields for backward compatibility

Phase 2: Migrate Existing Data​

  1. Projects:

    • If jiraKey exists, create a jiraProjects[] entry
    • If bitbucketRepos[] exists, create repositories[] entries
    • Mark first entry as isPrimary: true

  2. Persons:

    • If jiraAccountId exists, create externalAccounts[] entry
    • If bitbucketUsername exists, create externalAccounts[] entry
    • Mark as isPrimary: true

Phase 3: Update Sync Logic​

  1. Update sync operations to use link arrays
  2. Add link validation
  3. Add auto-inference suggestions
  4. Add link management UI

Phase 4: Deprecate Legacy Fields​

  1. Mark legacy fields as deprecated
  2. Provide migration path
  3. Eventually remove legacy fields

API Endpoints​

GET    /api/projects/[projectKey]/links
POST   /api/projects/[projectKey]/links/jira
POST   /api/projects/[projectKey]/links/repository
PUT    /api/projects/[projectKey]/links/[linkId]
DELETE /api/projects/[projectKey]/links/[linkId]
POST   /api/projects/[projectKey]/links/suggestions
GET    /api/persons/[personId]/links
POST   /api/persons/[personId]/links
PUT    /api/persons/[personId]/links/[linkId]
DELETE /api/persons/[personId]/links/[linkId]
POST   /api/persons/[personId]/links/suggestions
POST   /api/links/validate
POST   /api/links/verify

Benefits​

  1. Deterministic: No ambiguity about what links to what
  2. Flexible: Handle edge cases (multiple accounts, renamed projects)
  3. Auditable: Clear history of who linked what and when
  4. Maintainable: Easy to update links when external systems change
  5. Scalable: Support multiple external systems per entity

Implementation Checklist​

  • Update Project type with enhanced link structures
  • Update Person type with externalAccounts array
  • Create link management UI components
  • Implement auto-inference logic
  • Update sync operations to use links
  • Add link validation
  • Migrate existing data
  • Add link management API endpoints
  • Update documentation
  • Add link audit logging