Support Controlled Documents import (#7541)
Some checks failed
CI / build (push) Has been cancelled
CI / uitest (push) Has been cancelled
CI / uitest-pg (push) Has been cancelled
CI / uitest-qms (push) Has been cancelled
CI / svelte-check (push) Has been cancelled
CI / formatting (push) Has been cancelled
CI / test (push) Has been cancelled
CI / docker-build (push) Has been cancelled
CI / dist-build (push) Has been cancelled

* Support Controlled Documents import

Signed-off-by: Anna Khismatullina <anna.khismatullina@gmail.com>

* Add example table

Signed-off-by: Anna Khismatullina <anna.khismatullina@gmail.com>

---------

Signed-off-by: Anna Khismatullina <anna.khismatullina@gmail.com>
This commit is contained in:
Anna Khismatullina 2024-12-29 00:41:31 +07:00 committed by GitHub
parent d9604b7e0e
commit ffbd356cc2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
12 changed files with 1439 additions and 187 deletions

View File

@ -31,12 +31,24 @@ workspace/
│ └── files/
│ └── diagram.png # Can be referenced in markdown content
└── Project Alpha.yaml # Project configuration
├── QMS Documents/ # QMS documentation
│ ├── [SOP-001] Document Control.md # Document template
│ ├── [SOP-001] Document Control/ # Template implementations
│ │ └── [SOP-002] Document Review.md # Controlled document
│ └── [WI-001] Document Template Usage.md # Standalone controlled document
└── QMS Documents.yaml # QMS space configuration
```
### File Format Requirements
* All spaces files must be in YAML format
* All document/issue files must include YAML frontmatter followed by Markdown content
* Children documents/issues are located in the folder with the same name as the parent document/issue
#### Space Configuration (*.yaml)
Project space (`Project Alpha.yaml`):
#### Tracker Issues
##### 1. Project Configuration (*.yaml)
Example: `Project Alpha.yaml`:
```yaml
class: tracker:class:Project # Required
title: Project Alpha # Required
@ -51,32 +63,8 @@ description: string # Optional
defaultIssueStatus: Todo # Optional
```
Teamspace (`Documentation.yaml`):
```yaml
class: document:class:Teamspace # Required
title: Documentation # Required
private: false # Optional, default: false
autoJoin: true # Optional, default: true
owners: # Optional, list of email addresses
- john.doe@example.com
members: # Optional, list of email addresses
- joe.shmoe@example.com
description: string # Optional
```
#### Documents and Issues (*.md)
All files must include YAML frontmatter followed by Markdown content:
Document (`Getting Started.md`):
```yaml
---
class: document:class:Document # Required
title: Getting Started Guide # Required
---
# Content in Markdown format
```
Issue (`1.Project Setup.md`):
##### 2. Issue (*.md)
Example: `1.Project Setup.md`:
```yaml
---
class: tracker:class:Issue # Required
@ -90,11 +78,11 @@ remainingTime: 4 # Optional, in hours
Task description in Markdown...
```
### Task Identification
* Human-readable task ID is formed by combining project's identifier and task number from filename
* Example: For project with identifier "ALPHA" and task "1.Setup Project.md", the task ID will be "ALPHA-1"
##### Issue Identification
* Human-readable issue ID is formed by combining project's identifier and issue number from filename
* Example: For project with identifier `ALPHA` and issue `1.Setup Project.md`, the issue ID will be `ALPHA-1`
### Allowed Values
##### Allowed Values
Issue status values:
* `Backlog`
@ -109,6 +97,99 @@ Issue priority values:
* `High`
* `Urgent`
#### Documents
##### 1. Teamspace Configuration (*.yaml)
Example: `Documentation.yaml`:
```yaml
class: document:class:Teamspace # Required
title: Documentation # Required
private: false # Optional, default: false
autoJoin: true # Optional, default: true
owners: # Optional, list of email addresses
- john.doe@example.com
members: # Optional, list of email addresses
- joe.shmoe@example.com
description: string # Optional
```
##### 2. Document (*.md)
Example: `Getting Started.md`:
```yaml
---
class: document:class:Document # Required
title: Getting Started Guide # Required
---
# Content in Markdown format
```
#### Controlled Documents
##### 1. Space Configuration (*.yaml)
QMS Document Space: `QMS Documents.yaml`:
```yaml
class: documents:class:OrgSpace # Required
title: QMS Documents # Required
private: false # Optional, default: false
owners: # Optional, list of email addresses
- john.doe@example.com
members: # Optional, list of email addresses
- joe.shmoe@example.com
description: string # Optional
qualified: john.doe@example.com # Optional, qualified user
manager: jane.doe@example.com # Optional, QMS manager
qara: bob.smith@example.com # Optional, QA/RA specialist
```
##### 2. Document Template (*.md)
Example: `[SOP-001] Document Control.md`:
```yaml
---
class: documents:mixin:DocumentTemplate # Required
title: SOP Template # Required
docPrefix: SOP # Required, document code prefix
category: documents:category:Procedures # Required
author: John Smith # Required
owner: Jane Wilson # Required
abstract: Template description # Optional
reviewers: # Optional
- alice.cooper@example.com
approvers: # Optional
- david.brown@example.com
coAuthors: # Optional
- bob.dylan@example.com
---
Template content in Markdown...
```
##### 3. Controlled Document (*.md)
Example: `[SOP-002] Document Review.md`:
```yaml
---
class: documents:class:ControlledDocument # Required
title: Document Review Procedure # Required
template: [SOP-001] Document Control.md # Required, path to template
author: John Smith # Required
owner: Jane Wilson # Required
abstract: Document description # Optional
reviewers: # Optional
- alice.cooper@example.com
approvers: # Optional
- david.brown@example.com
coAuthors: # Optional
- bob.dylan@example.com
changeControl: # Optional
description: Initial document creation
reason: Need for standardized process
impact: Improved document control
---
Document content in Markdown...
```
##### Controlled Document Code Format
* Document code must be specified in file name: `[CODE] Any File Name.md`
* If code is not specified for controlled document, it will be generated automatically using template's docPrefix and sequential number (e.g. `SOP-99`)
* If code is not specified for template, it will be generated automatically as `TMPL-seqNumber`, where `seqNumber` is the sequence number of the template in the space
### Run Import Tool
```bash
docker run \
@ -125,3 +206,6 @@ docker run \
* All users must exist in the system before import
* Assignees are mapped by full name
* Files in space directories can be used as attachments when referenced in markdown content
* Document codes (in square brackets) must be unique across all document spaces
* Controlled documents must be created in the same space as their templates
* Controlled documents can be imported only with `Draft` status

View File

@ -0,0 +1,11 @@
class: documents:class:OrgSpace
title: QMS Documents
description: Quality Management System Documentation
private: false
owners:
- user1
members:
- user1
qualified: user1
manager: user1
qara: user1

View File

@ -0,0 +1,32 @@
---
class: documents:mixin:DocumentTemplate
title: 'Standard Operating Procedure Template'
docPrefix: SOP
category: DOC
author: John Appleseed
owner: John Appleseed
abstract: Template for Standard Operating Procedures
reviewers:
- John Appleseed
approvers:
- John Appleseed
---
# Standard Operating Procedure
## 1. Purpose
[Describe the purpose of the procedure]
## 2. Scope
[Define the scope and applicability]
## 3. Responsibilities
[List key roles and responsibilities]
## 4. Procedure
[Detail the step-by-step procedure]
## 5. References
[List related documents]
## 6. Revision History
[Document revision history]

View File

@ -0,0 +1,42 @@
---
class: documents:class:ControlledDocument
title: Document Review Procedure
template: '../[SOP-001] Document Control.md'
author: John Appleseed
owner: John Appleseed
abstract: Procedure for document review and approval process
reviewers:
- John Appleseed
approvers:
- John Appleseed
changeControl:
description: Initial document creation
reason: Need for standardized review process
impact: Improved document quality control
---
# Document Review Procedure
## 1. Purpose
This procedure defines the process for reviewing quality management system documents.
## 2. Scope
Applies to all controlled documents within the QMS.
## 3. Responsibilities
- Document Owner: Responsible for content
- Reviewers: Technical review
- QA Manager: Final approval
## 4. Procedure
1. Author prepares document
2. Technical review
3. QA review
4. Final approval
5. Document release
## 5. References
- Quality Manual
- Document Control Procedure
## 6. Revision History
Rev 0.1 - Initial draft

View File

@ -0,0 +1,37 @@
---
class: documents:class:ControlledDocument
title: Document Template Usage Guide
template: '[SOP-001] Document Control.md'
author: John Appleseed
owner: John Appleseed
abstract: Work instruction for using document templates
reviewers:
- John Appleseed
approvers:
- John Appleseed
---
# Document Template Usage Guide
## 1. Purpose
Guide users in proper usage of QMS document templates.
## 2. Scope
All personnel creating QMS documentation.
## 3. Procedure
1. Select appropriate template
2. Fill in required sections
3. Submit for review
## 4. Document Review Changes
| Step | Current Text | Updated Text | Comments |
| --- | --- | --- | --- |
| Initial Review | Select a template from library | Select a template that matches your document type | Clarified selection criteria |
| Metadata | Fill in required fields | Complete all required metadata and content fields | Added metadata specification |
| Review Process | Submit for review | Submit document for review according to procedure | Added reference to procedure |
| Approval | Wait for approval | Submit for approval after receiving all reviews | Process detail added |
## 5. References
- [Document Control SOP](./[SOP-001]%20Document%20Control.md)
- [Document Review Procedure](./[SOP-001]%20Document%20Control/[SOP-002]%20Document%20Review.md)

View File

@ -15,7 +15,7 @@
import { documentsId } from '@hcengineering/controlled-documents'
import documents from '@hcengineering/controlled-documents-resources/src/plugin'
import type { Client, Doc, Ref, Role } from '@hcengineering/core'
import type { Client, Doc, Ref } from '@hcengineering/core'
import { type ObjectSearchCategory, type ObjectSearchFactory } from '@hcengineering/model-presentation'
import { mergeIds, type Resource } from '@hcengineering/platform'
import { type TagCategory } from '@hcengineering/tags'
@ -71,11 +71,6 @@ export default mergeIds(documentsId, documents, {
TableDocumentTemplate: '' as Ref<Doc>,
TableDocumentDomain: '' as Ref<Doc>
},
role: {
QARA: '' as Ref<Role>,
Manager: '' as Ref<Role>,
QualifiedUser: '' as Ref<Role>
},
notification: {
DocumentsNotificationGroup: '' as Ref<NotificationGroup>,
ContentNotification: '' as Ref<NotificationType>,

View File

@ -45,6 +45,7 @@
"@hcengineering/chunter": "^0.6.20",
"@hcengineering/collaboration": "^0.6.0",
"@hcengineering/contact": "^0.6.24",
"@hcengineering/controlled-documents": "^0.1.0",
"@hcengineering/core": "^0.6.32",
"@hcengineering/document": "^0.6.0",
"@hcengineering/model-attachment": "^0.6.0",

View File

@ -14,7 +14,7 @@
//
import { type Attachment } from '@hcengineering/attachment'
import contact, { type Person, type PersonAccount } from '@hcengineering/contact'
import contact, { Employee, type Person, type PersonAccount } from '@hcengineering/contact'
import { type Class, type Doc, generateId, type Ref, type Space, type TxOperations } from '@hcengineering/core'
import document, { type Document } from '@hcengineering/document'
import { MarkupMarkType, type MarkupNode, MarkupNodeType, traverseNode, traverseNodeMarks } from '@hcengineering/text'
@ -28,6 +28,8 @@ import { ImportWorkspaceBuilder } from '../importer/builder'
import {
type ImportAttachment,
type ImportComment,
ImportControlledDocument,
ImportControlledDocumentTemplate,
type ImportDocument,
ImportDrawing,
type ImportIssue,
@ -35,11 +37,18 @@ import {
type ImportProjectType,
type ImportTeamspace,
type ImportWorkspace,
WorkspaceImporter
WorkspaceImporter,
ImportOrgSpace
} from '../importer/importer'
import { type Logger } from '../importer/logger'
import { BaseMarkdownPreprocessor } from '../importer/preprocessor'
import { type FileUploader } from '../importer/uploader'
import documents, {
DocumentState,
DocumentCategory,
ControlledDocument,
DocumentMeta
} from '@hcengineering/controlled-documents'
interface UnifiedComment {
author: string
@ -59,7 +68,7 @@ interface UnifiedIssueHeader {
}
interface UnifiedSpaceSettings {
class: 'tracker:class:Project' | 'document:class:Teamspace'
class: 'tracker:class:Project' | 'document:class:Teamspace' | 'documents:class:OrgSpace'
title: string
private?: boolean
autoJoin?: boolean
@ -101,13 +110,53 @@ interface UnifiedWorkspaceSettings {
}>
}
interface UnifiedChangeControlHeader {
description?: string
reason?: string
impact?: string
}
interface UnifiedControlledDocumentHeader {
class: 'documents:class:ControlledDocument'
title: string
template: string
author: string
owner: string
abstract?: string
reviewers?: string[]
approvers?: string[]
coAuthors?: string[]
changeControl?: UnifiedChangeControlHeader
}
interface UnifiedDocumentTemplateHeader {
class: 'documents:mixin:DocumentTemplate'
title: string
category: string
docPrefix: string
author: string
owner: string
abstract?: string
reviewers?: string[]
approvers?: string[]
coAuthors?: string[]
changeControl?: UnifiedChangeControlHeader
}
interface UnifiedOrgSpaceSettings extends UnifiedSpaceSettings {
class: 'documents:class:OrgSpace'
qualified?: string
manager?: string
qara?: string
}
class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
constructor (
private readonly urlProvider: (id: string) => string,
private readonly logger: Logger,
private readonly metadataByFilePath: Map<string, DocMetadata>,
private readonly metadataById: Map<Ref<Doc>, DocMetadata>,
private readonly attachMetadataByPath: Map<string, AttachmentMetadata>,
private readonly pathById: Map<Ref<Doc>, string>,
private readonly refMetaByPath: Map<string, ReferenceMetadata>,
private readonly attachMetaByPath: Map<string, AttachmentMetadata>,
personsByName: Map<string, Ref<Person>>
) {
super(personsByName)
@ -130,18 +179,24 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
const src = node.attrs?.src
if (src === undefined) return
const sourceMeta = this.getSourceMetadata(id)
if (sourceMeta == null) return
const sourcePath = this.getSourcePath(id)
if (sourcePath == null) return
const href = decodeURI(src as string)
const fullPath = path.resolve(path.dirname(sourceMeta.path), href)
const attachmentMeta = this.attachMetadataByPath.get(fullPath)
const fullPath = path.resolve(path.dirname(sourcePath), href)
const attachmentMeta = this.attachMetaByPath.get(fullPath)
if (attachmentMeta === undefined) {
this.logger.error(`Attachment image not found for ${fullPath}`)
return
}
const sourceMeta = this.refMetaByPath.get(sourcePath)
if (sourceMeta === undefined) {
this.logger.error(`Source metadata not found for ${sourcePath}`)
return
}
this.updateAttachmentMetadata(fullPath, attachmentMeta, id, spaceId, sourceMeta)
this.alterImageNode(node, attachmentMeta.id, attachmentMeta.name)
}
@ -150,22 +205,25 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
traverseNodeMarks(node, (mark) => {
if (mark.type !== MarkupMarkType.link) return
const sourceMeta = this.getSourceMetadata(id)
if (sourceMeta == null) return
const sourcePath = this.getSourcePath(id)
if (sourcePath == null) return
const href = decodeURI(mark.attrs.href)
const fullPath = path.resolve(path.dirname(sourceMeta.path), href)
const fullPath = path.resolve(path.dirname(sourcePath), href)
if (this.metadataByFilePath.has(fullPath)) {
const targetDocMeta = this.metadataByFilePath.get(fullPath)
if (this.refMetaByPath.has(fullPath)) {
const targetDocMeta = this.refMetaByPath.get(fullPath)
if (targetDocMeta !== undefined) {
this.alterInternalLinkNode(node, targetDocMeta)
}
} else if (this.attachMetadataByPath.has(fullPath)) {
const attachmentMeta = this.attachMetadataByPath.get(fullPath)
} else if (this.attachMetaByPath.has(fullPath)) {
const attachmentMeta = this.attachMetaByPath.get(fullPath)
if (attachmentMeta !== undefined) {
this.alterAttachmentLinkNode(node, attachmentMeta)
this.updateAttachmentMetadata(fullPath, attachmentMeta, id, spaceId, sourceMeta)
const sourceMeta = this.refMetaByPath.get(sourcePath)
if (sourceMeta !== undefined) {
this.updateAttachmentMetadata(fullPath, attachmentMeta, id, spaceId, sourceMeta)
}
}
} else {
this.logger.log('Unknown link type, leave it as is: ' + href)
@ -192,7 +250,7 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
}
}
private alterInternalLinkNode (node: MarkupNode, targetMeta: DocMetadata): void {
private alterInternalLinkNode (node: MarkupNode, targetMeta: ReferenceMetadata): void {
node.type = MarkupNodeType.reference
node.attrs = {
id: targetMeta.id,
@ -223,13 +281,13 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
return mimeType !== false ? mimeType : undefined
}
private getSourceMetadata (id: Ref<Doc>): DocMetadata | null {
const sourceMeta = this.metadataById.get(id)
if (sourceMeta == null) {
this.logger.error(`Source metadata not found for ${id}`)
private getSourcePath (id: Ref<Doc>): string | null {
const sourcePath = this.pathById.get(id)
if (sourcePath == null) {
this.logger.error(`Source file path not found for ${id}`)
return null
}
return sourceMeta
return sourcePath
}
private updateAttachmentMetadata (
@ -237,9 +295,9 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
attachmentMeta: AttachmentMetadata,
id: Ref<Doc>,
spaceId: Ref<Space>,
sourceMeta: DocMetadata
sourceMeta: ReferenceMetadata
): void {
this.attachMetadataByPath.set(fullPath, {
this.attachMetaByPath.set(fullPath, {
...attachmentMeta,
spaceId,
parentId: id,
@ -248,10 +306,9 @@ class HulyMarkdownPreprocessor extends BaseMarkdownPreprocessor {
}
}
interface DocMetadata {
interface ReferenceMetadata {
id: Ref<Doc>
class: string
path: string
refTitle: string
}
@ -265,12 +322,14 @@ interface AttachmentMetadata {
}
export class UnifiedFormatImporter {
private readonly metadataById = new Map<Ref<Doc>, DocMetadata>()
private readonly metadataByFilePath = new Map<string, DocMetadata>()
private readonly fileMetadataByPath = new Map<string, AttachmentMetadata>()
private readonly pathById = new Map<Ref<Doc>, string>()
private readonly refMetaByPath = new Map<string, ReferenceMetadata>()
private readonly fileMetaByPath = new Map<string, AttachmentMetadata>()
private readonly ctrlDocTemplateIdByPath = new Map<string, Ref<ControlledDocument>>()
private personsByName = new Map<string, Ref<Person>>()
private accountsByEmail = new Map<string, Ref<PersonAccount>>()
private employeesByName = new Map<string, Ref<Employee>>()
constructor (
private readonly client: TxOperations,
@ -278,14 +337,18 @@ export class UnifiedFormatImporter {
private readonly logger: Logger
) {}
async importFolder (folderPath: string): Promise<void> {
private async initCaches (): Promise<void> {
await this.cachePersonsByNames()
await this.cacheAccountsByEmails()
await this.cacheEmployeesByName()
}
async importFolder (folderPath: string): Promise<void> {
await this.initCaches()
const workspaceData = await this.processImportFolder(folderPath)
await this.collectFileMetadata(folderPath)
const workspaceData = await this.processImportFolder(folderPath)
this.logger.log('========================================')
this.logger.log('IMPORT DATA STRUCTURE: ' + JSON.stringify(workspaceData))
this.logger.log('========================================')
@ -294,9 +357,9 @@ export class UnifiedFormatImporter {
const preprocessor = new HulyMarkdownPreprocessor(
this.fileUploader.getFileUrl,
this.logger,
this.metadataByFilePath,
this.metadataById,
this.fileMetadataByPath,
this.pathById,
this.refMetaByPath,
this.fileMetaByPath,
this.personsByName
)
await new WorkspaceImporter(
@ -310,7 +373,7 @@ export class UnifiedFormatImporter {
this.logger.log('Importing attachments...')
const attachments: ImportAttachment[] = await Promise.all(
Array.from(this.fileMetadataByPath.values())
Array.from(this.fileMetaByPath.values())
.filter((attachMeta) => attachMeta.parentId !== undefined)
.map(async (attachMeta: AttachmentMetadata) => await this.processAttachment(attachMeta))
)
@ -433,6 +496,15 @@ export class UnifiedFormatImporter {
break
}
case documents.class.OrgSpace: {
const orgSpace = await this.processOrgSpace(spaceConfig as UnifiedOrgSpaceSettings)
builder.addOrgSpace(spacePath, orgSpace)
if (fs.existsSync(spacePath) && fs.statSync(spacePath).isDirectory()) {
await this.processControlledDocumentsRecursively(builder, spacePath, spacePath)
}
break
}
default: {
throw new Error(`Unknown space class ${spaceConfig.class} in ${spaceName}`)
}
@ -468,15 +540,13 @@ export class UnifiedFormatImporter {
const numberMatch = issueFile.match(/^(\d+)\./)
const issueNumber = numberMatch?.[1]
const meta: DocMetadata = {
const meta: ReferenceMetadata = {
id: generateId<Issue>(),
class: tracker.class.Issue,
path: issuePath,
refTitle: `${projectIdentifier}-${issueNumber}`
}
this.metadataById.set(meta.id, meta)
this.metadataByFilePath.set(issuePath, meta)
this.pathById.set(meta.id, issuePath)
this.refMetaByPath.set(issuePath, meta)
const issue: ImportIssue = {
id: meta.id as Ref<Issue>,
@ -525,6 +595,14 @@ export class UnifiedFormatImporter {
return account
}
private findEmployeeByName (name: string): Ref<Employee> {
const employee = this.employeesByName.get(name)
if (employee === undefined) {
throw new Error(`Employee not found: ${name}`)
}
return employee
}
private async processDocumentsRecursively (
builder: ImportWorkspaceBuilder,
teamspacePath: string,
@ -543,15 +621,14 @@ export class UnifiedFormatImporter {
}
if (docHeader.class === document.class.Document) {
const docMeta: DocMetadata = {
const docMeta: ReferenceMetadata = {
id: generateId<Document>(),
class: document.class.Document,
path: docPath,
refTitle: docHeader.title
}
this.metadataById.set(docMeta.id, docMeta)
this.metadataByFilePath.set(docPath, docMeta)
this.pathById.set(docMeta.id, docPath)
this.refMetaByPath.set(docPath, docMeta)
const doc: ImportDocument = {
id: docMeta.id as Ref<Document>,
@ -574,6 +651,79 @@ export class UnifiedFormatImporter {
}
}
private async processControlledDocumentsRecursively (
builder: ImportWorkspaceBuilder,
spacePath: string,
currentPath: string,
parentDocPath?: string
): Promise<void> {
const docFiles = fs.readdirSync(currentPath).filter((f) => f.endsWith('.md'))
for (const docFile of docFiles) {
const docPath = path.join(currentPath, docFile)
const docHeader = (await this.readYamlHeader(docPath)) as
| UnifiedControlledDocumentHeader
| UnifiedDocumentTemplateHeader
if (docHeader.class === undefined) {
this.logger.error(`Skipping ${docFile}: not a document`)
continue
}
if (
docHeader.class !== documents.class.ControlledDocument &&
docHeader.class !== documents.mixin.DocumentTemplate
) {
throw new Error(`Unknown document class ${docHeader.class} in ${docFile}`)
}
const documentMetaId = generateId<DocumentMeta>()
const refMeta: ReferenceMetadata = {
id: documentMetaId,
class: documents.class.DocumentMeta,
refTitle: docHeader.title
}
this.refMetaByPath.set(docPath, refMeta)
if (docHeader.class === documents.class.ControlledDocument) {
const docId = generateId<ControlledDocument>()
this.pathById.set(docId, docPath)
const doc = await this.processControlledDocument(
docHeader as UnifiedControlledDocumentHeader,
docPath,
docId,
documentMetaId
)
builder.addControlledDocument(spacePath, docPath, doc, parentDocPath)
} else {
if (!this.ctrlDocTemplateIdByPath.has(docPath)) {
const templateId = generateId<ControlledDocument>()
this.ctrlDocTemplateIdByPath.set(docPath, templateId)
this.pathById.set(templateId, docPath)
}
const templateId = this.ctrlDocTemplateIdByPath.get(docPath)
if (templateId === undefined) {
throw new Error(`Template ID not found: ${docPath}`)
}
const template = await this.processControlledDocumentTemplate(
docHeader as UnifiedDocumentTemplateHeader,
docPath,
templateId,
documentMetaId
)
builder.addControlledDocumentTemplate(spacePath, docPath, template, parentDocPath)
}
const subDir = path.join(currentPath, docFile.replace('.md', ''))
if (fs.existsSync(subDir) && fs.statSync(subDir).isDirectory()) {
await this.processControlledDocumentsRecursively(builder, spacePath, subDir, docPath)
}
}
}
private processComments (currentPath: string, comments: UnifiedComment[] = []): Promise<ImportComment[]> {
return Promise.all(
comments.map(async (comment) => {
@ -581,7 +731,7 @@ export class UnifiedFormatImporter {
if (comment.attachments !== undefined) {
for (const attachmentPath of comment.attachments) {
const fullPath = path.resolve(currentPath, attachmentPath)
const attachmentMeta = this.fileMetadataByPath.get(fullPath)
const attachmentMeta = this.fileMetaByPath.get(fullPath)
if (attachmentMeta !== undefined) {
const importAttachment = await this.processAttachment(attachmentMeta)
attachments.push(importAttachment)
@ -650,6 +800,114 @@ export class UnifiedFormatImporter {
}
}
private async processOrgSpace (spaceHeader: UnifiedOrgSpaceSettings): Promise<ImportOrgSpace> {
return {
class: documents.class.OrgSpace,
title: spaceHeader.title,
private: spaceHeader.private ?? false,
archived: spaceHeader.archived ?? false,
description: spaceHeader.description,
owners: spaceHeader.owners?.map((email) => this.findAccountByEmail(email)) ?? [],
members: spaceHeader.members?.map((email) => this.findAccountByEmail(email)) ?? [],
qualified: spaceHeader.qualified !== undefined ? this.findAccountByEmail(spaceHeader.qualified) : undefined,
manager: spaceHeader.manager !== undefined ? this.findAccountByEmail(spaceHeader.manager) : undefined,
qara: spaceHeader.qara !== undefined ? this.findAccountByEmail(spaceHeader.qara) : undefined,
docs: []
}
}
private async processControlledDocument (
header: UnifiedControlledDocumentHeader,
docPath: string,
id: Ref<ControlledDocument>,
metaId: Ref<DocumentMeta>
): Promise<ImportControlledDocument> {
const codeMatch = path.basename(docPath).match(/^\[([^\]]+)\]/)
const author = this.findEmployeeByName(header.author)
const owner = this.findEmployeeByName(header.owner)
if (author === undefined || owner === undefined) {
throw new Error(`Author or owner not found: ${header.author} or ${header.owner}`)
}
const templatePath = path.resolve(path.dirname(docPath), header.template)
if (!fs.existsSync(templatePath)) {
throw new Error(`Template file not found: ${templatePath}`)
}
if (!this.ctrlDocTemplateIdByPath.has(templatePath)) {
const templateId = generateId<ControlledDocument>()
this.ctrlDocTemplateIdByPath.set(templatePath, templateId)
this.pathById.set(templateId, templatePath)
}
const templateId = this.ctrlDocTemplateIdByPath.get(templatePath)
if (templateId === undefined) {
throw new Error(`Template ID not found: ${templatePath}`)
}
return {
id,
metaId,
class: documents.class.ControlledDocument,
title: header.title,
template: templateId,
code: codeMatch?.[1],
major: 0,
minor: 1,
state: DocumentState.Draft,
author,
owner,
abstract: header.abstract,
reviewers: header.reviewers?.map((email) => this.findEmployeeByName(email)) ?? [],
approvers: header.approvers?.map((email) => this.findEmployeeByName(email)) ?? [],
coAuthors: header.coAuthors?.map((email) => this.findEmployeeByName(email)) ?? [],
descrProvider: async () => await this.readMarkdownContent(docPath),
ccReason: header.changeControl?.reason,
ccImpact: header.changeControl?.impact,
ccDescription: header.changeControl?.description,
subdocs: []
}
}
private async processControlledDocumentTemplate (
header: UnifiedDocumentTemplateHeader,
docPath: string,
id: Ref<ControlledDocument>,
metaId: Ref<DocumentMeta>
): Promise<ImportControlledDocumentTemplate> {
const author = this.findEmployeeByName(header.author)
const owner = this.findEmployeeByName(header.owner)
if (author === undefined || owner === undefined) {
throw new Error(`Author or owner not found: ${header.author} or ${header.owner}`)
}
const codeMatch = path.basename(docPath).match(/^\[([^\]]+)\]/)
return {
id,
metaId,
class: documents.mixin.DocumentTemplate,
title: header.title,
docPrefix: header.docPrefix,
code: codeMatch?.[1],
major: 0,
minor: 1,
state: DocumentState.Draft,
category: header.category as Ref<DocumentCategory>,
author,
owner,
abstract: header.abstract,
reviewers: header.reviewers?.map((email) => this.findEmployeeByName(email)) ?? [],
approvers: header.approvers?.map((email) => this.findEmployeeByName(email)) ?? [],
coAuthors: header.coAuthors?.map((email) => this.findEmployeeByName(email)) ?? [],
descrProvider: async () => await this.readMarkdownContent(docPath),
ccReason: header.changeControl?.reason,
ccImpact: header.changeControl?.impact,
ccDescription: header.changeControl?.description,
subdocs: []
}
}
private async readYamlHeader (filePath: string): Promise<any> {
this.logger.log('Read YAML header from: ' + filePath)
const content = fs.readFileSync(filePath, 'utf8')
@ -688,6 +946,20 @@ export class UnifiedFormatImporter {
}, new Map())
}
private async cacheEmployeesByName (): Promise<void> {
this.employeesByName = (await this.client.findAll(contact.mixin.Employee, {}))
.map((employee) => {
return {
_id: employee._id,
name: employee.name.split(',').reverse().join(' ')
}
})
.reduce((refByName, employee) => {
refByName.set(employee.name, employee._id)
return refByName
}, new Map())
}
private async collectFileMetadata (folderPath: string): Promise<void> {
const processDir = async (dir: string): Promise<void> => {
const entries = fs.readdirSync(dir, { withFileTypes: true })
@ -699,7 +971,7 @@ export class UnifiedFormatImporter {
await processDir(fullPath)
} else if (entry.isFile()) {
const attachmentId = generateId<Attachment>()
this.fileMetadataByPath.set(fullPath, { id: attachmentId, name: entry.name, path: fullPath })
this.fileMetaByPath.set(fullPath, { id: attachmentId, name: entry.name, path: fullPath })
}
}
}

View File

@ -12,10 +12,15 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//
import documents, { ControlledDocument, DocumentState } from '@hcengineering/controlled-documents'
import { type DocumentQuery, type Ref, type Status, type TxOperations } from '@hcengineering/core'
import document from '@hcengineering/document'
import tracker, { IssuePriority, type IssueStatus } from '@hcengineering/tracker'
import {
ImportControlledDocument,
ImportControlledDocumentTemplate,
ImportOrgSpace,
type ImportControlledDoc,
type ImportDocument,
type ImportIssue,
type ImportProject,
@ -39,15 +44,21 @@ const PROJECT_IDENTIFIER_REGEX = /^[a-zA-Z_][a-zA-Z0-9_]*$/
export class ImportWorkspaceBuilder {
private readonly projects = new Map<string, ImportProject>()
private readonly teamspaces = new Map<string, ImportTeamspace>()
private readonly projectTypes = new Map<string, ImportProjectType>()
private readonly issuesByProject = new Map<string, Map<string, ImportIssue>>()
private readonly issueParents = new Map<string, string>()
private readonly teamspaces = new Map<string, ImportTeamspace>()
private readonly documentsByTeamspace = new Map<string, Map<string, ImportDocument>>()
private readonly documentParents = new Map<string, string>()
private readonly errors = new Map<string, ValidationError>()
private readonly qmsSpaces = new Map<string, ImportOrgSpace>()
private readonly qmsTemplates = new Map<Ref<ControlledDocument>, string>()
private readonly qmsDocsBySpace = new Map<string, Map<string, ImportControlledDoc>>()
private readonly qmsDocsParents = new Map<string, string>()
private readonly projectTypes = new Map<string, ImportProjectType>()
private readonly issueStatusCache = new Map<string, Ref<IssueStatus>>()
private readonly errors = new Map<string, ValidationError>()
constructor (
private readonly client: TxOperations,
@ -125,10 +136,95 @@ export class ImportWorkspaceBuilder {
return this
}
addOrgSpace (path: string, space: ImportOrgSpace): this {
this.validateAndAdd('documentSpace', path, space, (s) => this.validateOrgSpace(s), this.qmsSpaces, path)
return this
}
addControlledDocument (
spacePath: string,
docPath: string,
doc: ImportControlledDocument,
parentDocPath?: string
): this {
if (!this.qmsDocsBySpace.has(spacePath)) {
this.qmsDocsBySpace.set(spacePath, new Map())
}
const docs = this.qmsDocsBySpace.get(spacePath)
if (docs === undefined) {
throw new Error(`Document space ${spacePath} not found`)
}
if (doc.code !== undefined) {
const duplicateDoc = Array.from(docs.values()).find((existingDoc) => existingDoc.code === doc.code)
if (duplicateDoc !== undefined) {
throw new Error(`Duplicate document code ${doc.code} in space ${spacePath}`)
}
}
this.validateAndAdd(
'controlledDocument',
docPath,
doc,
(d) => this.validateControlledDocument(d as ImportControlledDocument),
docs,
docPath
)
if (parentDocPath !== undefined) {
this.qmsDocsParents.set(docPath, parentDocPath)
}
return this
}
addControlledDocumentTemplate (
spacePath: string,
templatePath: string,
template: ImportControlledDocumentTemplate,
parentTemplatePath?: string
): this {
if (!this.qmsDocsBySpace.has(spacePath)) {
this.qmsDocsBySpace.set(spacePath, new Map())
}
const qmsDocs = this.qmsDocsBySpace.get(spacePath)
if (qmsDocs === undefined) {
throw new Error(`Document space ${spacePath} not found`)
}
if (template.code !== undefined) {
const duplicate = Array.from(qmsDocs.values()).find((existingDoc) => existingDoc.code === template.code)
if (duplicate !== undefined) {
throw new Error(`Duplicate document code ${template.code} in space ${spacePath}`)
}
}
this.validateAndAdd(
'documentTemplate',
templatePath,
template,
(t) => this.validateControlledDocumentTemplate(t as ImportControlledDocumentTemplate),
qmsDocs,
templatePath
)
if (parentTemplatePath !== undefined) {
this.qmsDocsParents.set(templatePath, parentTemplatePath)
}
if (template.id !== undefined) {
this.qmsTemplates.set(template.id, templatePath)
}
return this
}
validate (): ValidationResult {
// Perform cross-entity validation
this.validateProjectReferences()
this.validateSpaceDocuments()
this.validateSpacesReferences()
this.validateDocumentsReferences()
return {
isValid: this.errors.size === 0,
@ -173,9 +269,27 @@ export class ImportWorkspaceBuilder {
}
}
for (const [spacePath, qmsDocs] of this.qmsDocsBySpace) {
const space = this.qmsSpaces.get(spacePath)
if (space !== undefined) {
const rootDocPaths = Array.from(qmsDocs.keys()).filter((docPath) => !this.qmsDocsParents.has(docPath))
for (const rootPath of rootDocPaths) {
this.buildControlledDocumentHierarchy(rootPath, qmsDocs)
}
space.docs = rootDocPaths.map((path) => qmsDocs.get(path)).filter(Boolean) as ImportControlledDocument[]
}
}
return {
projectTypes: Array.from(this.projectTypes.values()),
spaces: [...Array.from(this.projects.values()), ...Array.from(this.teamspaces.values())]
spaces: [
...Array.from(this.projects.values()),
...Array.from(this.teamspaces.values()),
...Array.from(this.qmsSpaces.values())
],
attachments: []
}
}
@ -415,7 +529,7 @@ export class ImportWorkspaceBuilder {
return errors
}
private validateProjectReferences (): void {
private validateSpacesReferences (): void {
// Validate project type references
for (const project of this.projects.values()) {
if (project.projectType !== undefined && !this.projectTypes.has(project.projectType.name)) {
@ -424,7 +538,7 @@ export class ImportWorkspaceBuilder {
}
}
private validateSpaceDocuments (): void {
private validateDocumentsReferences (): void {
// Validate that issues belong to projects and documents to teamspaces
for (const projectPath of this.issuesByProject.keys()) {
if (!this.projects.has(projectPath)) {
@ -437,6 +551,23 @@ export class ImportWorkspaceBuilder {
this.addError(teamspacePath, 'Documents reference non-existent teamspace')
}
}
for (const [orgSpacePath, docs] of this.qmsDocsBySpace) {
if (!this.qmsSpaces.has(orgSpacePath)) {
this.addError(orgSpacePath, 'Controlled document reference non-existent orgSpace')
}
for (const [docPath, doc] of docs) {
if (doc.class === documents.class.ControlledDocument) {
const templateRef = (doc as ImportControlledDocument).template
const templatePath = this.qmsTemplates.get(templateRef)
if (templatePath === undefined) {
this.addError(docPath, 'Controlled document reference non-existent template')
} else if (!docs.has(templatePath)) {
this.addError(docPath, 'Controlled document reference not in space')
}
}
}
}
}
private addError (path: string, error: string): void {
@ -471,6 +602,20 @@ export class ImportWorkspaceBuilder {
issue.subdocs = childIssues
}
private buildControlledDocumentHierarchy (docPath: string, allDocs: Map<string, ImportControlledDoc>): void {
const doc = allDocs.get(docPath)
if (doc === undefined) return
const childDocs = Array.from(allDocs.entries())
.filter(([childPath]) => this.qmsDocsParents.get(childPath) === docPath)
.map(([childPath, childDoc]) => {
this.buildControlledDocumentHierarchy(childPath, allDocs)
return childDoc
})
doc.subdocs = childDocs
}
private validateEmoji (emoji: string): string[] {
const errors: string[] = []
if (typeof emoji === 'string' && emoji.codePointAt(0) == null) {
@ -529,4 +674,165 @@ export class ImportWorkspaceBuilder {
}
return errors
}
private validateOrgSpace (space: ImportOrgSpace): string[] {
const errors: string[] = []
if (space.class !== documents.class.OrgSpace) {
errors.push('Invalid space class: ' + space.class)
}
errors.push(...this.validateType(space.title, 'string', 'title'))
if (space.emoji !== undefined) {
errors.push(...this.validateEmoji(space.emoji))
}
if (space.owners !== undefined) {
errors.push(...this.validateArray(space.owners, 'string', 'owners'))
}
if (space.members !== undefined) {
errors.push(...this.validateArray(space.members, 'string', 'members'))
}
return errors
}
private validateControlledDocument (doc: ImportControlledDocument): string[] {
const errors: string[] = []
// Validate required fields presence and types
errors.push(...this.validateType(doc.title, 'string', 'title'))
errors.push(...this.validateType(doc.class, 'string', 'class'))
errors.push(...this.validateType(doc.template, 'string', 'template'))
errors.push(...this.validateType(doc.state, 'string', 'state'))
if (doc.code !== undefined) {
errors.push(...this.validateType(doc.code, 'string', 'code'))
}
// Validate required string fields are defined
if (!this.validateStringDefined(doc.title)) errors.push('title is required')
if (!this.validateStringDefined(doc.template)) errors.push('template is required')
// Validate numbers are positive
if (!this.validatePossitiveNumber(doc.major)) errors.push('invalid value for field "major"')
if (!this.validatePossitiveNumber(doc.minor)) errors.push('invalid value for field "minor"')
// Validate arrays
errors.push(...this.validateArray(doc.reviewers, 'string', 'reviewers'))
errors.push(...this.validateArray(doc.approvers, 'string', 'approvers'))
errors.push(...this.validateArray(doc.coAuthors, 'string', 'coAuthors'))
// Validate optional fields if present
if (doc.author !== undefined) {
errors.push(...this.validateType(doc.author, 'string', 'author'))
}
if (doc.owner !== undefined) {
errors.push(...this.validateType(doc.owner, 'string', 'owner'))
}
if (doc.abstract !== undefined) {
errors.push(...this.validateType(doc.abstract, 'string', 'abstract'))
}
if (doc.ccDescription !== undefined) {
errors.push(...this.validateType(doc.ccDescription, 'string', 'ccDescription'))
}
if (doc.ccImpact !== undefined) {
errors.push(...this.validateType(doc.ccImpact, 'string', 'ccImpact'))
}
if (doc.ccReason !== undefined) {
errors.push(...this.validateType(doc.ccReason, 'string', 'ccReason'))
}
// Validate class
if (doc.class !== documents.class.ControlledDocument) {
errors.push('invalid class: ' + doc.class)
}
// Validate state values
if (doc.state !== DocumentState.Draft) {
errors.push('invalid state: ' + doc.state)
}
// todo: validate seqNumber is not duplicated (unique prefix? code?)
return errors
}
private validateControlledDocumentTemplate (template: ImportControlledDocumentTemplate): string[] {
const errors: string[] = []
// Validate required fields presence and types
errors.push(...this.validateType(template.title, 'string', 'title'))
errors.push(...this.validateType(template.class, 'string', 'class'))
errors.push(...this.validateType(template.docPrefix, 'string', 'docPrefix'))
errors.push(...this.validateType(template.state, 'string', 'state'))
if (template.code !== undefined) {
errors.push(...this.validateType(template.code, 'string', 'code'))
}
// Validate required string fields are defined
if (!this.validateStringDefined(template.title)) errors.push('title is required')
if (!this.validateStringDefined(template.docPrefix)) errors.push('docPrefix is required')
// Validate numbers are positive
if (!this.validatePossitiveNumber(template.major)) errors.push('invalid value for field "major"')
if (!this.validatePossitiveNumber(template.minor)) errors.push('invalid value for field "minor"')
// Validate arrays
errors.push(...this.validateArray(template.reviewers, 'string', 'reviewers'))
errors.push(...this.validateArray(template.approvers, 'string', 'approvers'))
errors.push(...this.validateArray(template.coAuthors, 'string', 'coAuthors'))
// Validate optional fields if present
if (template.author !== undefined) {
errors.push(...this.validateType(template.author, 'string', 'author'))
}
if (template.owner !== undefined) {
errors.push(...this.validateType(template.owner, 'string', 'owner'))
}
if (template.abstract !== undefined) {
errors.push(...this.validateType(template.abstract, 'string', 'abstract'))
}
if (template.ccDescription !== undefined) {
errors.push(...this.validateType(template.ccDescription, 'string', 'ccDescription'))
}
if (template.ccImpact !== undefined) {
errors.push(...this.validateType(template.ccImpact, 'string', 'ccImpact'))
}
if (template.ccReason !== undefined) {
errors.push(...this.validateType(template.ccReason, 'string', 'ccReason'))
}
// Validate class
if (template.class !== documents.mixin.DocumentTemplate) {
errors.push('invalid class: ' + template.class)
}
// Validate state values
if (template.state !== DocumentState.Draft) {
errors.push('invalid state: ' + template.state)
}
// todo: validate seqNumber no duplicated
return errors
}
private validateControlledDocumentSpaces (): void {
// Validate document spaces
for (const [spacePath] of this.qmsSpaces) {
// Validate controlled documents
const docs = this.qmsDocsBySpace.get(spacePath)
if (docs !== undefined) {
// for (const [docPath, doc] of docs) {
for (const docPath of docs.keys()) {
// Check parent document exists
const parentPath = this.documentParents.get(docPath)
if (parentPath !== undefined && !docs.has(parentPath)) {
this.addError(docPath, `Parent document not found: ${parentPath}`)
}
}
}
}
}
}

View File

@ -14,12 +14,25 @@
//
import attachment, { Drawing, type Attachment } from '@hcengineering/attachment'
import chunter, { type ChatMessage } from '@hcengineering/chunter'
import { type Person } from '@hcengineering/contact'
import { Employee, type Person } from '@hcengineering/contact'
import documents, {
ChangeControl,
type ControlledDocument,
createControlledDocMetadata,
createDocumentTemplateMetadata,
DocumentCategory,
DocumentMeta,
type DocumentSpace,
DocumentState,
DocumentTemplate,
OrgSpace,
ProjectDocument,
useDocumentTemplate
} from '@hcengineering/controlled-documents'
import core, {
type Account,
type AttachedData,
type Class,
type Blob as PlatformBlob,
type CollaborativeDoc,
type Data,
type Doc,
@ -27,7 +40,9 @@ import core, {
generateId,
makeCollabId,
type Mixin,
type Blob as PlatformBlob,
type Ref,
RolesAssignment,
SortingOrder,
type Space,
type Status,
@ -157,6 +172,57 @@ export interface ImportDrawing {
contentProvider: () => Promise<string>
}
export type ImportControlledDoc = ImportControlledDocument | ImportControlledDocumentTemplate // todo: rename
export interface ImportOrgSpace extends ImportSpace<ImportControlledDoc> {
class: Ref<Class<DocumentSpace>>
qualified?: Ref<Account>
manager?: Ref<Account>
qara?: Ref<Account>
}
export interface ImportControlledDocumentTemplate extends ImportDoc {
id: Ref<ControlledDocument>
metaId: Ref<DocumentMeta>
class: Ref<Class<Document>>
docPrefix: string
code?: string
major: number
minor: number
state: DocumentState
category?: Ref<DocumentCategory>
author?: Ref<Employee>
owner?: Ref<Employee>
abstract?: string
reviewers?: Ref<Employee>[]
approvers?: Ref<Employee>[]
coAuthors?: Ref<Employee>[]
ccReason?: string
ccImpact?: string
ccDescription?: string
subdocs: ImportControlledDoc[]
}
export interface ImportControlledDocument extends ImportDoc {
id: Ref<ControlledDocument>
metaId: Ref<DocumentMeta>
class: Ref<Class<ControlledDocument>>
template: Ref<ControlledDocument> // todo: test (it was Ref<DocumentTemplate>)
code?: string
major: number
minor: number
state: DocumentState
reviewers?: Ref<Employee>[]
approvers?: Ref<Employee>[]
coAuthors?: Ref<Employee>[]
author?: Ref<Employee>
owner?: Ref<Employee>
abstract?: string
ccReason?: string
ccImpact?: string
ccDescription?: string
subdocs: ImportControlledDoc[]
}
export class WorkspaceImporter {
private readonly issueStatusByName = new Map<string, Ref<IssueStatus>>()
private readonly projectTypeByName = new Map<string, Ref<ProjectType>>()
@ -192,6 +258,8 @@ export class WorkspaceImporter {
await this.importTeamspace(space as ImportTeamspace)
} else if (space.class === tracker.class.Project) {
await this.importProject(space as ImportProject)
} else if (space.class === documents.class.OrgSpace) {
await this.importOrgSpace(space as ImportOrgSpace)
}
}
}
@ -730,4 +798,333 @@ export class WorkspaceImporter {
}
return identifier
}
async importOrgSpace (space: ImportOrgSpace): Promise<Ref<DocumentSpace>> {
this.logger.log('Creating document space: ' + space.title)
const spaceId = await this.createOrgSpace(space)
this.logger.log('Document space created: ' + spaceId)
// Create hierarchy meta
const templateMetaMap = new Map<Ref<ControlledDocument>, { seqNumber: number, code: string }>()
for (const doc of space.docs) {
if (this.isDocumentTemplate(doc)) {
await this.createDocTemplateMetaHierarhy(doc as ImportControlledDocumentTemplate, templateMetaMap, spaceId)
} else {
await this.createControlledDocMetaHierarhy(doc as ImportControlledDocument, templateMetaMap, spaceId)
}
}
// Partition templates and documents
const templateMap = new Map<Ref<ControlledDocument>, ImportControlledDocumentTemplate>()
const documentMap = new Map<Ref<ControlledDocument>, ImportControlledDocument>()
for (const doc of space.docs) {
this.partitionTemplatesFromDocuments(doc, documentMap, templateMap)
}
// Create attached docs for templates
for (const template of templateMap.values()) {
const meta = templateMetaMap.get(template.id)
if (meta === undefined) {
throw new Error('Template meta not found: ' + template.id)
}
await this.createDocTemplateAttachedDoc(template, meta.seqNumber, meta.code, spaceId)
}
// Create attached docs for documents
for (const document of documentMap.values()) {
await this.createControlledDocAttachedDoc(document, spaceId)
}
return spaceId
}
private partitionTemplatesFromDocuments (
doc: ImportControlledDoc,
documentMap: Map<Ref<ControlledDocument>, ImportControlledDocument>,
templateMap: Map<Ref<ControlledDocument>, ImportControlledDocumentTemplate>
): void {
if (this.isDocumentTemplate(doc)) {
templateMap.set(doc.id, doc as ImportControlledDocumentTemplate)
} else {
documentMap.set(doc.id, doc as ImportControlledDocument)
}
for (const subdoc of doc.subdocs) {
this.partitionTemplatesFromDocuments(subdoc, documentMap, templateMap)
}
}
private isDocumentTemplate (doc: ImportDoc): boolean {
return doc.class === documents.mixin.DocumentTemplate
}
private async createOrgSpace (space: ImportOrgSpace): Promise<Ref<DocumentSpace>> {
const spaceId = generateId<DocumentSpace>()
const data: Data<OrgSpace> = {
type: documents.spaceType.DocumentSpaceType,
description: space.description ?? '',
name: space.title,
private: space.private,
owners: space.owners ?? [],
members: space.members ?? [],
archived: space.archived ?? false
}
await this.client.createDoc(documents.class.OrgSpace, core.space.Space, data, spaceId)
const rolesAssignment: RolesAssignment = {}
if (space.qualified !== undefined) {
rolesAssignment[documents.role.QualifiedUser] = [space.qualified]
}
if (space.manager !== undefined) {
rolesAssignment[documents.role.Manager] = [space.manager]
}
if (space.qara !== undefined) {
rolesAssignment[documents.role.QARA] = [space.qara]
}
if (Object.keys(rolesAssignment).length > 0) {
await this.client.createMixin(
spaceId,
documents.class.OrgSpace,
core.space.Space,
documents.mixin.DocumentSpaceTypeData,
rolesAssignment
)
}
return spaceId
}
private async createDocTemplateMetaHierarhy (
template: ImportControlledDocumentTemplate,
templateMetaMap: Map<Ref<ControlledDocument>, { seqNumber: number, code: string }>,
spaceId: Ref<DocumentSpace>,
parentProjectDocumentId?: Ref<ProjectDocument>
): Promise<Ref<ControlledDocument>> {
this.logger.log('Creating document template: ' + template.title)
const templateId = template.id ?? generateId<ControlledDocument>()
const { seqNumber, code, projectDocumentId } = await createDocumentTemplateMetadata(
this.client,
documents.class.Document,
spaceId,
documents.mixin.DocumentTemplate,
undefined,
parentProjectDocumentId,
templateId as unknown as Ref<ControlledDocument>, // todo: suspisios place
template.docPrefix,
template.code ?? '',
template.title,
template.metaId
)
templateMetaMap.set(templateId, { seqNumber, code })
for (const subdoc of template.subdocs) {
if (this.isDocumentTemplate(subdoc)) {
await this.createDocTemplateMetaHierarhy(
subdoc as ImportControlledDocumentTemplate,
templateMetaMap,
spaceId,
projectDocumentId
)
} else {
await this.createControlledDocMetaHierarhy(
subdoc as ImportControlledDocument,
templateMetaMap,
spaceId,
projectDocumentId
)
}
}
return templateId
}
private async createDocTemplateAttachedDoc (
template: ImportControlledDocumentTemplate,
seqNumber: number,
code: string,
spaceId: Ref<DocumentSpace>
): Promise<Ref<ControlledDocument>> {
const content = await template.descrProvider()
this.logger.log('Creating document template attached doc: ' + template.title)
const collabId = makeCollabId(documents.class.Document, template.id, 'content')
const contentId = await this.createCollaborativeContent(template.id, collabId, content, spaceId)
const changeControlId =
template.ccReason !== undefined || template.ccImpact !== undefined || template.ccDescription !== undefined
? await this.createChangeControl(spaceId, template.ccDescription, template.ccReason, template.ccImpact)
: ('' as Ref<ChangeControl>)
const ops = this.client.apply()
const result = await ops.addCollection(
documents.class.ControlledDocument,
spaceId,
template.metaId,
documents.class.DocumentMeta,
'documents',
{
title: template.title,
major: template.major,
minor: template.minor,
state: template.state,
author: template.author,
owner: template.owner,
abstract: template.abstract,
reviewers: template.reviewers ?? [],
approvers: template.approvers ?? [],
coAuthors: template.coAuthors ?? [],
code,
seqNumber,
prefix: template.docPrefix, // todo: or TEMPLATE_PREFIX?s
content: contentId,
changeControl: changeControlId,
commentSequence: 0,
requests: 0,
labels: 0
},
template.id as unknown as Ref<ControlledDocument> // todo: make sure it's not used anywhere as mixin id
)
await ops.createMixin(template.id, documents.class.Document, spaceId, documents.mixin.DocumentTemplate, {
sequence: 0,
docPrefix: template.docPrefix
})
const commit = await ops.commit()
if (!commit.result) {
throw new Error('Failed to create document template attached doc: ' + template.title)
}
this.logger.log('Document template attached doc created: ' + result)
return result
}
private async createControlledDocMetaHierarhy (
doc: ImportControlledDocument,
templateMetaMap: Map<Ref<ControlledDocument>, { seqNumber: number, code: string }>,
spaceId: Ref<DocumentSpace>,
parentProjectDocumentId?: Ref<ProjectDocument>
): Promise<Ref<ControlledDocument>> {
this.logger.log('Creating controlled document: ' + doc.title)
const documentId = doc.id ?? generateId<ControlledDocument>()
// const { seqNumber, prefix, category } = await useDocumentTemplate(this.client, doc.template as unknown as Ref<DocumentTemplate>)
const result = await createControlledDocMetadata(
this.client,
documents.template.ProductChangeControl, // todo: make it dynamic - wtf, commit missed?
documentId,
spaceId,
undefined, // project
parentProjectDocumentId, // parent
'prefix',
0,
doc.code ?? '',
doc.title,
doc.metaId
)
// Process subdocs recursively
for (const subdoc of doc.subdocs) {
if (this.isDocumentTemplate(subdoc)) {
await this.createDocTemplateMetaHierarhy(
subdoc as ImportControlledDocumentTemplate,
templateMetaMap,
spaceId,
result.projectDocumentId
)
} else {
await this.createControlledDocMetaHierarhy(
subdoc as ImportControlledDocument,
templateMetaMap,
spaceId,
result.projectDocumentId
)
}
}
return documentId
}
private async createControlledDocAttachedDoc (
document: ImportControlledDocument,
spaceId: Ref<DocumentSpace>
): Promise<Ref<ControlledDocument>> {
this.logger.log('Creating controlled document attached doc: ' + document.title)
const content = await document.descrProvider()
const collabId = makeCollabId(documents.class.Document, document.id, 'content')
const contentId = await this.createCollaborativeContent(document.id, collabId, content, spaceId)
const templateId = document.template
const { seqNumber, prefix, category } = await useDocumentTemplate(
this.client,
templateId as unknown as Ref<DocumentTemplate>
)
const ops = this.client.apply()
const changeControlId =
document.ccReason !== undefined || document.ccImpact !== undefined
? await this.createChangeControl(spaceId, document.ccDescription, document.ccReason, document.ccImpact)
: ('' as Ref<ChangeControl>)
const result = await ops.addCollection(
documents.class.ControlledDocument,
spaceId,
document.metaId,
documents.class.DocumentMeta,
'documents',
{
title: document.title,
major: document.major,
minor: document.minor,
state: document.state,
author: document.author,
owner: document.owner,
abstract: document.abstract,
reviewers: document.reviewers ?? [],
approvers: document.approvers ?? [],
coAuthors: document.coAuthors ?? [],
changeControl: changeControlId,
code: document.code ?? `${prefix}-${seqNumber}`,
prefix,
category,
seqNumber,
content: contentId,
template: templateId as unknown as Ref<DocumentTemplate>,
commentSequence: 0,
requests: 0
},
document.id
)
await ops.updateDoc(documents.class.DocumentMeta, spaceId, document.metaId, {
documents: 0,
title: `${prefix}-${seqNumber} ${document.title}`
})
await ops.commit()
this.logger.log('Controlled document attached doc created: ' + result)
return result
}
private async createChangeControl (
spaceId: Ref<DocumentSpace>,
description?: string,
reason?: string,
impact?: string
): Promise<Ref<ChangeControl>> {
const changeControlData: Data<ChangeControl> = {
reason: reason ?? '',
impact: impact ?? '',
description: description ?? '',
impactedDocuments: []
}
return await this.client.createDoc(documents.class.ChangeControl, spaceId, changeControlData)
}
}

View File

@ -16,16 +16,16 @@
import { type Employee } from '@hcengineering/contact'
import { type AttachedData, type Class, type Ref, type TxOperations, Blob, Mixin } from '@hcengineering/core'
import {
type Document,
type DocumentTemplate,
type ControlledDocument,
type Document,
type DocumentCategory,
type DocumentSpace,
type DocumentMeta,
type DocumentSpace,
type DocumentTemplate,
type HierarchyDocument,
type Project,
DocumentState,
HierarchyDocument,
ProjectDocument
type ProjectDocument,
DocumentState
} from './types'
import documents from './plugin'
@ -67,18 +67,55 @@ export async function createControlledDocFromTemplate (
return { seqNumber: -1, success: false }
}
const { seqNumber, prefix, content, category } = await useDocumentTemplate(client, templateId)
const { success, documentMetaId } = await createControlledDocMetadata(
client,
templateId,
documentId,
space,
project,
parent,
prefix,
seqNumber,
spec.code,
spec.title
)
if (!success) {
return { seqNumber: -1, success: false }
}
await client.addCollection(
docClass,
space,
documentMetaId,
documents.class.DocumentMeta,
'documents',
{
...spec,
category,
template: templateId,
seqNumber,
prefix,
state: DocumentState.Draft,
content
},
documentId
)
return { seqNumber, success: true }
}
export async function useDocumentTemplate (
client: TxOperations,
templateId: Ref<DocumentTemplate>
): Promise<{ seqNumber: number, prefix: string, content: Ref<Blob> | null, category: Ref<DocumentCategory> }> {
const template = await client.findOne(documents.mixin.DocumentTemplate, {
_id: templateId
})
if (template === undefined) {
return { seqNumber: -1, success: false }
}
let path: Array<Ref<DocumentMeta>> = []
if (parent !== undefined) {
path = await getParentPath(client, parent)
return { seqNumber: -1, prefix: '', content: null, category: '' as Ref<DocumentCategory> }
}
await client.updateMixin(templateId, documents.class.Document, template.space, documents.mixin.DocumentTemplate, {
@ -89,34 +126,27 @@ export async function createControlledDocFromTemplate (
const seqNumber = template.sequence + 1
const prefix = template.docPrefix
return await createControlledDoc(
client,
templateId,
documentId,
{ ...spec, category: template.category },
space,
project,
prefix,
seqNumber,
path,
docClass,
template.content
)
return { seqNumber, prefix, content: template.content, category: template.category as Ref<DocumentCategory> }
}
async function createControlledDoc (
export async function createControlledDocMetadata (
client: TxOperations,
templateId: Ref<DocumentTemplate>,
documentId: Ref<ControlledDocument>,
spec: AttachedData<ControlledDocument>,
space: Ref<DocumentSpace>,
project: Ref<Project> | undefined,
parent: Ref<ProjectDocument> | undefined,
prefix: string,
seqNumber: number,
path: Ref<DocumentMeta>[] = [],
docClass: Ref<Class<ControlledDocument>> = documents.class.ControlledDocument,
content: Ref<Blob> | null
): Promise<{ seqNumber: number, success: boolean }> {
specCode: string,
specTitle: string,
metaId?: Ref<DocumentMeta>
): Promise<{
success: boolean
seqNumber: number
documentMetaId: Ref<DocumentMeta>
projectDocumentId: Ref<ProjectDocument>
}> {
const projectId = project ?? documents.ids.NoProject
const ops = client.apply()
@ -127,23 +157,33 @@ async function createControlledDoc (
})
ops.notMatch(documents.class.Document, {
code: spec.code
code: specCode
})
const metaId = await ops.createDoc(documents.class.DocumentMeta, space, {
documents: 0,
title: `${prefix}-${seqNumber} ${spec.title}`
})
const documentMetaId = await ops.createDoc(
documents.class.DocumentMeta,
space,
{
documents: 0,
title: `${prefix}-${seqNumber} ${specTitle}`
},
metaId
)
let path: Array<Ref<DocumentMeta>> = []
if (parent !== undefined) {
path = await getParentPath(client, parent)
}
const projectMetaId = await ops.createDoc(documents.class.ProjectMeta, space, {
project: projectId,
meta: metaId,
meta: documentMetaId,
path,
parent: path[0] ?? documents.ids.NoParent,
documents: 0
})
await client.addCollection(
const projectDocumentId = await client.addCollection(
documents.class.ProjectDocument,
space,
projectMetaId,
@ -156,25 +196,9 @@ async function createControlledDoc (
}
)
await ops.addCollection(
docClass,
space,
metaId,
documents.class.DocumentMeta,
'documents',
{
...spec,
template: templateId,
seqNumber,
prefix,
state: DocumentState.Draft,
content
},
documentId
)
const success = await ops.commit()
return { seqNumber, success: success.result }
return { success: success.result, seqNumber, documentMetaId, projectDocumentId }
}
export async function createDocumentTemplate (
@ -190,6 +214,70 @@ export async function createDocumentTemplate (
category: Ref<DocumentCategory>,
author?: Ref<Employee>
): Promise<{ seqNumber: number, success: boolean }> {
const { success, seqNumber, code, documentMetaId } = await createDocumentTemplateMetadata(
client,
_class,
space,
_mixin,
project,
parent,
templateId,
prefix,
spec.code ?? '',
spec.title
)
if (!success) {
return { seqNumber: -1, success: false }
}
const ops = client.apply()
await ops.addCollection<DocumentMeta, HierarchyDocument>(
_class,
space,
documentMetaId,
documents.class.DocumentMeta,
'documents',
{
...spec,
code,
seqNumber,
category,
prefix: TEMPLATE_PREFIX,
author,
owner: author,
content: spec.content ?? null
},
templateId
)
await ops.createMixin(templateId, documents.class.Document, space, _mixin, {
sequence: 0,
docPrefix: prefix
})
const commit = await ops.commit()
return { seqNumber, success: commit.result }
}
export async function createDocumentTemplateMetadata (
client: TxOperations,
_class: Ref<Class<Document>>,
space: Ref<DocumentSpace>,
_mixin: Ref<Mixin<DocumentTemplate>>,
project: Ref<Project> | undefined,
parent: Ref<ProjectDocument> | undefined,
templateId: Ref<ControlledDocument>,
prefix: string,
specCode: string,
specTitle: string,
metaId?: Ref<DocumentMeta>
): Promise<{
success: boolean
seqNumber: number
code: string
documentMetaId: Ref<DocumentMeta>
projectDocumentId: Ref<ProjectDocument>
}> {
const projectId = project ?? documents.ids.NoProject
const incResult = await client.updateDoc(
@ -202,7 +290,7 @@ export async function createDocumentTemplate (
true
)
const seqNumber = (incResult as any).object.sequence as number
const code = spec.code === '' ? `${TEMPLATE_PREFIX}-${seqNumber}` : spec.code
const code = specCode === '' ? `${TEMPLATE_PREFIX}-${seqNumber}` : specCode
let path: Array<Ref<DocumentMeta>> = []
@ -225,20 +313,25 @@ export async function createDocumentTemplate (
docPrefix: prefix
})
const metaId = await ops.createDoc(documents.class.DocumentMeta, space, {
documents: 0,
title: `${TEMPLATE_PREFIX}-${seqNumber} ${spec.title}`
})
const documentMetaId = await ops.createDoc(
documents.class.DocumentMeta,
space,
{
documents: 0,
title: `${TEMPLATE_PREFIX}-${seqNumber} ${specTitle}`
},
metaId
)
const projectMetaId = await ops.createDoc(documents.class.ProjectMeta, space, {
project: projectId,
meta: metaId,
meta: documentMetaId,
path,
parent: path[0] ?? documents.ids.NoParent,
documents: 0
})
await client.addCollection(
const projectDocumentId = await client.addCollection(
documents.class.ProjectDocument,
space,
projectMetaId,
@ -251,31 +344,7 @@ export async function createDocumentTemplate (
}
)
await ops.addCollection<DocumentMeta, HierarchyDocument>(
_class,
space,
metaId,
documents.class.DocumentMeta,
'documents',
{
...spec,
code,
seqNumber,
category,
prefix: TEMPLATE_PREFIX,
author,
owner: author,
content: null
},
templateId
)
await ops.createMixin(templateId, documents.class.Document, space, _mixin, {
sequence: 0,
docPrefix: prefix
})
const success = await ops.commit()
return { seqNumber, success: success.result }
return { success: success.result, seqNumber, code, documentMetaId, projectDocumentId }
}

View File

@ -6,7 +6,8 @@ import {
type Type,
type Space,
type SpaceTypeDescriptor,
type Permission
type Permission,
Role
} from '@hcengineering/core'
import type { Asset, Plugin, Resource } from '@hcengineering/platform'
import { IntlString, plugin } from '@hcengineering/platform'
@ -276,6 +277,11 @@ export const documentsPlugin = plugin(documentsId, {
CA: '' as Ref<DocumentCategory>,
CC: '' as Ref<DocumentCategory>
},
role: {
QARA: '' as Ref<Role>,
Manager: '' as Ref<Role>,
QualifiedUser: '' as Ref<Role>
},
resolver: {
Location: '' as Resource<(loc: Location) => Promise<ResolvedLocation | undefined>>
},